Pull to refresh

Development

Show first
Rating limit
Level of difficulty

The Rules for Data Processing Pipeline Builders

Reading time5 min
Views3.8K


"Come, let us make bricks, and burn them thoroughly."
– legendary builders

You may have noticed by 2020 that data is eating the world. And whenever any reasonable amount of data needs processing, a complicated multi-stage data processing pipeline will be involved.


At Bumble — the parent company operating Badoo and Bumble apps — we apply hundreds of data transforming steps while processing our data sources: a high volume of user-generated events, production databases and external systems. This all adds up to quite a complex system! And just as with any other engineering system, unless carefully maintained, pipelines tend to turn into a house of cards — failing daily, requiring manual data fixes and constant monitoring.


For this reason, I want to share certain good engineering practises with you, ones that make it possible to build scalable data processing pipelines from composable steps. While some engineers understand such rules intuitively, I had to learn them by doing, making mistakes, fixing, sweating and fixing things again…


So behold! I bring you my favourite Rules for Data Processing Pipeline Builders.

Read more →

Development of “YaRyadom” (“I’mNear”) application under the control of Vk Mini Apps. Part 1 .Net Core

Reading time8 min
Views1K
Application is developed in order to help people find their peers who share similar interests and to be able to spend some time doing what you like. The project is currently on the stage of beta-testing in the social network “VKontakte”. Right now I am in the process of fixing bugs and adding everything that is missing. I felt like I could use a bit of destruction and decided to write a little about the development. While I was writing, I decided to divide the text into different parts. Here we are going to pay more attention to backend nuances which I faced, and to everything that a user does not see.
Read more →

Configuring FT4232H using the ftdi_eeprom

Reading time2 min
Views8.1K


The FT4232H is USB 2.0 High speed to UART IC converter. The FT4232H has four UART ports and one USB port.


By connecting EEPROM memory to this chip, you can set specific operating modes or change the manufacturer's data.


Let's look at the example and configure FT4232H directly on a system running GNU/Linux. We will do this using the ftdi_eeprom.

Read more →

Algorithms in Go: Sliding Window Pattern

Reading time3 min
Views5.9K

Let's consider the following problem: we have an array of integers and we need to find out the length of the smallest subarray the sum of which is no less than the target number. If we don't have such a subarray we shall return -1.

We can start with a naive approach and consider every possible subarray in the input:

Continue reading

Playing with Nvidia's New Ampere GPUs and Trying MIG

Reading time11 min
Views4.4K


Every time when the essential question arises, whether to upgrade the cards in the server room or not, I look through similar articles and watch such videos.


Channel with the aforementioned video is very underestimated, but the author does not deal with ML. In general, when analyzing comparisons of accelerators for ML, several things usually catch your eye:


  • The authors usually take into account only the "adequacy" for the market of new cards in the United States;
  • The ratings are far from the people and are made on very standard networks (which is probably good overall) without details;
  • The popular mantra to train more and more gigantic models makes its own adjustments to the comparison;

The answer to the question "which card is better?" is not rocket science: Cards of the 20* series didn't get much popularity, while the 1080 Ti from Avito (Russian craigslist) still are very attractive (and, oddly enough, don't get cheaper, probably for this reason).


All this is fine and dandy and the standard benchmarks are unlikely to lie too much, but recently I learned about the existence of Multi-Instance-GPU technology for A100 video cards and native support for TF32 for Ampere devices and I got the idea to share my experience of the real testing cards on the Ampere architecture (3090 and A100). In this short note, I will try to answer the questions:


  • Is the upgrade to Ampere worth it? (spoiler for the impatient — yes);
  • Are the A100 worth the money (spoiler — in general — no);
  • Are there any cases when the A100 is still interesting (spoiler — yes);
  • Is MIG technology useful (spoiler — yes, but for inference and for very specific cases for training);
Read more →

Russian AI Cup 2020 — a new strategy game for developers

Reading time5 min
Views2.5K


This year, many processes transformed, with traditions and habits being modified. The rhythm of life has changed, and there's more uncertainty and strain. But IT person's soul wants diversity, and many developers have asked us if annual Russian AI Cup will be held this year. Is there going to be an announcement? What is the main theme of the upcoming championship? Should I take a vacation?

Though some changes are expected, it will be held in keeping with the best traditions. In the run-up, we will announce one of today's largest online AI programming championships — Russian AI Cup. We invite you to make history!

Vital Characteristics Of The Best Webflow Designers

Reading time4 min
Views892
A great website serves as the main key to hit success. Enhancing the online presence of your brand is absolutely a must nowadays. Technological advancement has changed the global business landscape. Hence, it is important to have a webflow developer who will take charge of the codes to be used in website templates and designs. Webflow is a website tool. It is a flexible platform that is geared to create a homogenous biz site. The use of this tool will definitely pave the way for your brand to excel on the web.
Read more →

The Code Analyzer is wrong. Long live the Analyzer

Reading time6 min
Views715
Foo(std::move(buffer), line_buffer - buffer.get());

Combining many actions in a single C++ expression is a bad practice, as such code is hard to understand, maintain, and it is easy to make mistakes in it. For example, one can instill a bug by reconciling different actions when evaluating function arguments. We agree with the classic recommendation that code should be simple and clear. Now let's look at an interesting case where the PVS-Studio analyzer is technically wrong, but from a practical point of view, the code should still be changed.
Read more →

How static code analysis helps in the GameDev industry

Reading time7 min
Views1.4K
image1.png

The gaming industry is constantly evolving and is developing faster than a speeding bullet. Along with the growth of the industry, the complexity of development also increases: the code base is getting larger and the number of bugs is growing as well. Therefore, modern game projects need to pay special attention to the code quality. Today we will cover one of the ways to make your code more decent, which is static analysis, as well as how PVS-Studio in practice helps in the game project development of various sizes.
Read more →

Analyzing the Code Quality of Microsoft's Open XML SDK

Reading time10 min
Views958
image1.png

My first encounter with Open XML SDK took place when I was looking for a library that I could use to create some accounting documents in Word. After more than 7 years of working with Word API, I wanted to try something new and easier-to-use. That's how I learned that Microsoft offered an alternative solution. As tradition has it, before our team adopts any program or library, we check them with the PVS-Studio analyzer.
Read more →

How to build a high-performance application on Tarantool from scratch

Reading time33 min
Views3.1K
image

I came to Mail.ru Group in 2013, and I required a queue for one task. First of all, I decided to check what the company had already got. They told me they had this Tarantool product, and I checked how it worked and decided that adding a queue broker to it could work perfectly well.

I contacted Kostja Osipov, the senior expert in Tarantool, and the next day he gave me a 250-string script that was capable of managing almost everything I needed. Since that moment, I have been in love with Tarantool. It turned out that a small amount of code written with a quite simple script language was capable of ensuring some totally new performance for this DBMS.

Today, I’m going to tell you how to instantiate your own queue in Tarantool 2.2.
Read more →

Spring Boot app with Apache Kafka in Docker container

Reading time4 min
Views31K

Privet, comrads!

In this article i’ll show how easy it is to setup Spring Java app with Kafka message brocker. We will use docker containers for kafka zookeeper/brocker apps and configure plaintext authorization for access from both local and external net.

Link to final project on github can be picked up at the end of the article.

Read more

Active Termination Drivers

Reading time9 min
Views4K


The easiest way to build a driver with specified output impedance is to use an amplifier with high load compatibility and add a resistor to its output. The penalty is a voltage drop across this resistor, so there is power loss and we need a higher supply voltage. If our driver is able to deliver the same voltage and current to the same load, but the extra resistor will have a lower value, our device will be able to deliver the same output power at a lower supply voltage. Less power losses, less heat, and longer working time when a battery is used.
There is an idea how to solve this problem: active termination. We can synthesize the output impedance!

Now when we know what we want, go to design our drivers!
Read more →

Patroni cluster (with Zookeeper) in a docker swarm on a local machine

Reading time20 min
Views12K

There probably is no way one who stores some crucial data (and well, in particular, using SQL databases) can possibly dodge from thoughts of building some kind of safe cluster, distant guardian to protect consistency and availability at all times. Even if the main server with your precious database gets knocked out deadly - the show must go on, right? This basically means the database must still be available and data be up-to-date with the one on the failed server.

As you might have noticed, there are dozens of ways to go and Patroni is just one of them. There is plenty of articles providing a more or less detailed comparison of the options available, so I assume I'm free to skip the part of luring you into Patroni's side. Let's start off from the point where among others you are already leaning towards Patroni and are willing to try that out in a more or less real-case setup.

I am not a DevOps engineer originally so when the need for the high-availability cluster arose and I went on I would catch every single bump on the road. Hope this tutorial will help you out to get the job done with ease! If you don't want any more explanations, jump right in. Otherwise, you might want to read some more notes on the setup I went on with.

Read more

A Guide to App Monetization in China

Reading time5 min
Views1.8K

Introduction and Context

So you’ve just finished developing the next mobile gaming smash hit. You’ve already planned to launch your game worldwide and win the hearts of gamers everywhere. But what you haven’t done yet is think about how you’re going to make any money from your success and hard work. Of course, you’ve imagined becoming an instant millionaire, but you just don’t know how to get there.  

Fear not, intrepid developer! Instead of just rehashing the tired headlines and beating the same old drum, we’ve tailored our Ad Monetization write-up for developers who are gearing up to publish their game or app in China. We’ve scoured the internet, grabbed insight from ad giants like Google, Smartyads, and PocketGamer.biz and combined it with our own hands-on knowledge of what works in China—knowledge that we have gained from the past  5 years in the industry.

We need to briefly explore how monetization functions as a utility within the context of game design. This is especially true when examining how developers in China have made meaningful design decisions that have both shaped and been shaped by user habits. Unpacking the utility of Chinese game monetization will help developers understand why Ads Monetization or Hybrid Monetization strategies have become so popular, especially in China.

Moving from monetization design, we’ll examine some of the best practices regarding Ads Monetization. Finally,  we’ll turn to specific case studies in the Chinese market that show how powerful Ads Monetization, specifically Rewarded Videos, can be in a well-balanced monetization strategy. 

Designing Monetization as a Utility

Читать далее

Why PVS-Studio Doesn't Offer Automatic Fixes

Reading time4 min
Views769
Why PVS-Studio Doesn't Offer Automatic Fixes

Static analyzer PVS-Studio can detect bugs in pretty complex and intricate parts of code, and coming up with appropriate fixes for such bugs may be a tough task even for human developers. That's exactly the reason why we should avoid offering any options for automatic fixing at all. Here are a couple of examples.
Read more →

Jupyter for .NET. «Like Python»

Reading time2 min
Views2.1K
A few months ago Microsoft announced about the creation of Jupyter for .NET. However, people are barely interested in it despite how attractive the topic is. I decided to make a LaTeX wrapper for the Entity class from a symbolic algebra library:



Looks awesome. Is simple. Very enjoyable. Let's see more!
Read more →

Confusing extensions in Swift

Reading time4 min
Views2.2K
This post is a little bit the information aggregator. If you find a mistake, you could write to me about it I really appreciate that. Have a nice read.

Example with JSONDecoder


What would happen if we run the following piece of code?

struct Test<T>: Codable where T: Codable {
    enum CodingKeys: String, CodingKey {
        case value
    }
    
    let value: T
    let info: String
}

extension Test {
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        self.value = try container.decode(T.self, forKey: .value)
        self.info = "Default init(from decoder:)"
    }
}

extension Test where T == String {
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        self.value = try container.decode(T.self, forKey: .value)
        self.info = "Custom init(from decoder:)"
    }
}

let data = #"{"value":"Hello, World!"}"#.data(using: .utf8)!
let object = try? JSONDecoder().decode(Test<String>.self, from: data)
print(object.debugDescription)

Try thinking for 5 seconds about the result.

The result
Optional(
    Test<String>(
        value: "Hello, World!", 
        info: "Default init(from decoder:)"
    )
)


Read more →

What App Stores can you consider except Apple App Store and Google Play Store? Here are 6 Alternative App Stores

Reading time3 min
Views6.6K

Recently Epic Games’ Fortnite was removed from the Apple App Store and Google Play Store. The main reason being Epics bypassing of in-app purchases instead of using the officially sanctioned system for both platforms. While it is still possible for you to download Fortnite directly, this large scale case brings to light the duopoly of Apple and Google in the mobile market.

For most developers, these two stores account for almost all of their revenue and userbase. While Epic Games will be fine to go without, for the time being, what about the other 99% of developers who rely on these two stores for distributing and monetizing their apps. In this article, we’ll provide some of the alternative stores available for both developers and consumers for finding or distributing apps.

For one reason or another, you may have found yourself wondering, ‘where are some other places I can go to find and download new apps and games for my mobile device’? Or ‘are there any other marketplaces for me to share my apps’?

Читать далее