System Analysis and Design *

Analyze and project

KAPANDR Dec 29 2020 at 19:59

Tarantool: an analyst's view

8 min

1.9K

VK corporate blog System Analysis and Design *Database Administration *Internet marketing *Tarantool *

Hi all! I'm Andrey Kapustin. I work as a system analyst at Mail.ru Group. Our products form a unified ecosystem. Many independent infrastructures generate data in it: taxi and food delivery services, email services, social networks, etc. The faster and more precise we can predict a client's needs, the sooner and more correctly we can offer our products.

Many system analysts and engineers are keen to know:

How to design the architecture of a trigger platform for real-time marketing?
How to arrange a data structure that would be in line with the requirements of a marketing strategy for interacting with clients?
How to ensure the stable operations of the system under very heavy workloads?

Such systems are based on technologies of high-load processing and Big Data analysis. We have accumulated considerable experience in these areas. Our expertise is in high demand on the market. I'm going to show how we help our customers to switch from off-line to on-line in their interactions with clients using Real-Time Marketing solutions based on Tarantool.

+26

BitfuryRussia Feb 6 2019 at 21:00

Generic Methods in Rust: How Exonum Shifted from Iron to Actix-web

13 min

5.9K

Bitfury Group corporate blog Programming *System Analysis and Design *Rust *Distributed systems *

The Rust ecosystem is still growing. As a result, new libraries with improved functionality are frequently released into the developer community, while older libraries become obsolete. When we initially designed Exonum, we used the Iron web-framework. In this article, we describe how we ported the Exonum framework to actix-web using generic programming.

+26

danikin May 13 2019 at 12:02

Citymobil — a manual for improving availability amid business growth for startups. Part 5

8 min

VK corporate blog High performance *System Analysis and Design *Start-up development Product Management *

This is the final part of the series describing how we’re increasing our service availability in Citymobil (you can read the previous part here). Now I’m going to talk about one more type of outages and the conclusions we made about them, how we modified the development process, what automation we introduced.

+24

danikin Apr 22 2019 at 18:52

Citymobil — a manual for improving availability amid business growth for startups. Part 1

4 min

1.3K

VK corporate blog High performance *System Analysis and Design *Start-up development Product Management *

In this first part of an article series «Citymobil — a manual for improving availability amid business growth for startups» I’m going to break down the way we managed to dramatically scale up the availability of Citymobil services. The article opens with the story about our business, our task, the reason for this task to increase the availability emerged and limitations. Citymobil is a rapid-growing taxi aggregator. In 2018, it increased by more than 15 times in terms of number of successfully completed trips. Some months showed 50% increase compared with the previous month.

The business grew like a weed in every direction (it still does): there was an increase in server load, team size and number of deployments. At the same time the new threats to service availability emerged. The company faced a task of the most importance — how to increase availability without compromising company growth. In this article, I’ll talk about the way we managed to solve this task in a relatively short time.

+24

danikin Apr 29 2019 at 11:51

Citymobil — a manual for improving availability amid business growth for startups. Part 3

8 min

1.1K

VK corporate blog High performance *System Analysis and Design *Start-up development Product Management *

This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here and here). In further parts, I’ll talk about the accidents and outages in detail. But first let me highlight something I should’ve talked about in the first article but didn’t. I found out about it from my readers’ feedback. This article gives me a chance to fix this annoying shortcoming.

+23

Shapelez Aug 15 2019 at 00:22

Qrator filtering network configuration delivery system

6 min

1.3K

Qrator Labs corporate blog System Analysis and Design *IT Infrastructure *Network technologies *Distributed systems *

TL;DR: Client-server architecture of our internal configuration management tool, QControl.
At its basement, there’s a two-layered transport protocol working with gzip-compressed messages without decompression between endpoints. Distributed routers and endpoints receive the configuration updates, and the protocol itself makes it possible to install intermediary localized relays. It is based on a differential backup (“recent-stable,” explained further) design and employs JMESpath query language and Jinja templating for configuration rendering.

Qrator Labs operates on and maintains a globally distributed mitigation network. Our network is anycast, based on announcing our subnets via BGP. Being a BGP anycast network physically located in several regions across the Earth makes it possible for us to process and filter illegitimate traffic closer to the Internet backbone — Tier-1 operators.

On the other hand, being a geographically distributed network bears its difficulties. Communication between the network points-of-presence (PoP) is essential for a security provider to have a coherent configuration for all network nodes and update it in a timely and cohesive manner. So to provide the best possible service for customers, we had to find a way to synchronize the configuration data between different continents reliably.

In the beginning, there was the Word… which quickly became communication protocol in need of an upgrade.

+22

datacompboy Jan 15 2019 at 16:15

What to think during NALSD interview

7 min

9.1K

Google Developers corporate blog High performance *System Analysis and Design *IT career Distributed systems *

There are a lot of posts about what a typical coding interview at Google looks like. But, while not as widely described and discussed, there is also quite often a system design interview. For an SRE position it’s NALSD: non-abstract large system design. The key difference between SWE and SRE interviews consists in these two letters: NA.

Library in Google-Zurich

So, what is the difference? How to be prepared for this interview? Let’s be non-abstract, and use an example. To be more non-abstract, let’s take something from the material world, such that you won’t be asked the exact same thing at the real interview (at least, not at the Google interview) :)

So, let’s design a public library system. For the paper books, like you have seen everywhere around. The whole text below was written all at once within around one hour, to roughly show you the areas that you should be able to cover / touch during the interview. Please excuse some disorder, that’s how I think (therefore I am).

+22

danikin Apr 25 2019 at 12:03

Citymobil — a manual for improving availability amid business growth for startups. Part 2

8 min

977

VK corporate blog High performance *System Analysis and Design *Start-up development Product Management *

This is a second article out of a series «Citymobil — a manual for improving availability amid business growth for startups». You can read the first part here. Let’s continue to talk about the way we managed to improve the availability of Citymobil services. In the first article, we learned how to count the lost trips. Ok, we are counting them. What now? Now that we are equipped with an understandable tool to measure the lost trips, we can move to the most interesting part — how do we decrease losses? Without slowing down our current growth! Since it seemed to us that the lion’s share of technical problems causing the trips loss had something to do with the backend, we decided to turn our attention to the backend development process first. Jumping ahead of myself, I’m going to say that we were right — the backend became the main site of the battle for the lost trips.

+21

alextavgen Jan 15 2019 at 18:34

How to Painlessly Unite Art with Java, JavaScript, and Graphs or The Story Behind Creating an Interactive Theatre Produc

9 min

1.3K

Programming *Java *System Analysis and Design *VueJS *Cyberpunk

Last year 2018, a theatre production series called Tale of the Century was launched in Estonia. Throughout the year, 22 local theatres presented their interpretations of the past hundred years of Estonian history to the audiences. In the draw, the Russian Theatre was assigned the topic of the future of Estonia.

+16

danikin May 7 2019 at 12:43

Citymobil — a manual for improving availability amid business growth for startups. Part 4

7 min

VK corporate blog High performance *System Analysis and Design *Start-up development Product Management *

This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here: part 1, part 2, part 3). In further parts, I’ll talk about the accidents and outages in detail.

1. Bad release: database overload

Let me begin with a specific example of this type of outage. We deployed an optimization: added USE INDEX in an SQL query; during testing as well as in production, it sped up short queries, but the long ones — slowed down. The long queries slowdown was only noticed in production. As a result, a lot of long parallel queries caused the database to be down for an hour. We thoroughly studied the way USE INDEX worked; we described it in the Do’s and Dont’s file and warned the engineers against the incorrect usage. We also analyzed the query and realized that it retrieves mostly historical data and, therefore, can be run on a separate replica for historical requests. Even if this replica goes down due to an overload, the business will keep running.

+15

vovkins Jun 26 2020 at 11:31

Making a Tarantool-Based Investment Business Core for Alfa-Bank

10 min

1.8K

VK corporate blog High performance *System Analysis and Design *Database Administration *Tarantool *

A still from «Our Secret Universe: The Hidden Life of the Cell»

Investment business is one of the most complex domains in the banking world. It's about not just credits, loans, and deposits — there are also securities, currencies, commodities, derivatives, and all kinds of complex stuff like structured products.

Recently, people have become increasingly aware of their finances. More and more get involved in securities trading. Individual investment accounts have emerged not so long ago. They allow you to trade in securities and get tax credits or avoid taxes at the same time. All clients coming to us want to manage their portfolios and see their reporting on-line. Most frequently, these are multi-product portfolios, which means that people are clients of different business areas.

Moreover, the demands of regulators, both Russian and international, also grow.

To meet the current needs and lay a foundation for future upgrades, we've developed our Tarantool-based investment business core.

+14

Argon Jan 16 2019 at 00:25

Automation VS Chaos

5 min

1.1K

Programming *System Analysis and Design *Industrial Programming *Project management *

IT technologies evolution allowed to control huge data flows. Business has a lot of IT solutions: CRM, ERP, BPM, accounting systems or at least just Excel and Word. Companies are different too. Some of companies are composed of plenty branches. Let’s name such as “Pyramid”. Pyramids have data synchronization issue for pile of IT systems. Software vendors and versions differ for branches significantly. In addition management company continuously modify reporting requirements that causes frustration assaults in the branches. This is a story about the project I happened to encounter chaos that needed to be systematized and automated. Low budget and tight deadlines limited the use of most existing industrial solutions but opened up scope for creativity.

Читать дальше →

+12

BitfuryRussia Mar 10 2019 at 19:01

Crystal Blockchain Analytics: Investigating the Hacks and Theft Cases

8 min

2.7K

Bitfury Group corporate blog Information Security *System Analysis and Design *Distributed systems *Cryptocurrencies

In this report, Bitfury shares analysis completed by its Crystal Blockchain Analytics engineering team on the movement of bitcoin from the Zaif exchange, Bithumb exchange and Electrum wallets.

+11

potan Aug 14 2019 at 15:16

How to Catch a Cat with TLA+

3 min

1.9K

Waves corporate blog Programming *System Analysis and Design *Mathematics *Logic games

Many programmers struggle when using formal methods to solve problems within their programs, as those methods, while effective, can be unreasonably complex. To understand why this happens, let’s use the model checking method to solve a relatively easy puzzle:

Conditions

You’re in a hallway with seven doors on one side leading to seven rooms. A cat is hiding in one of these rooms. Your task is to catch the cat. Opening a door takes one step. If you guess the correct door, you catch the cat. If you do not guess the correct door, the cat runs to the next room.

NikZanyat Oct 1 2019 at 08:48

Quintet instead of Byte — data storage and retrieval approach

13 min

1.6K

Programming *System Analysis and Design *SQL *IT Standards *Data storage *

Quintet is a way to present atomic pieces of data indicating their role in the business area. Quintets can describe any item, while each of them contains complete information about itself and its relations to other quintets. Such description does not depend on the platform used. Its objective is to simplify the storage of data and to improve the visibility of their presentation.

We will discuss an approach to storing and processing information and share some thoughts on creating a development platform in this new paradigm. What for? To develop faster and in shorter iterations: sketch your project, make sure it is what you thought of, refine it, and then keep refining the result.

The quintet has properties: type, value, parent, and order among the peers. Thus, there are 5 components including the identifier. This is the simplest universal form to record information, a new standard that could potentially fit any programming demands. Quintets are stored in the file system of the unified structure, in a continuous homogeneous indexed bulk of data. The quintet data model — a data model that describes any data structure as a single interconnected list of basic types and terms based on them (metadata), as well as instances of objects stored according to this metadata (data).

wingedfox Mar 31 at 19:17

Use-Case Evolution Guide

Medium

13 min

444

System Analysis and Design *Designing and refactoring *Development Management *Project management *

Tutorial

Modern product development demands more and more sophisticated designs. This in turn leads to the increased complexity of both demand and implementation. Business is flooding the architecture and development teams with the new and changed requirements. Development teams are struggling to understand what the business demand is and find the best product increment strategy. One of the widely adopted conversational methods is the Use-Cases. This guide is intended to shed light on the process of the requirements development and maturing.

ryan0852 Sep 24 2019 at 11:14

Tutorial For Creating Blockchain Solution on Hyperledger Composer

7 min

2.9K

Decentralized networks *System Analysis and Design *Development of mobile applications *Mobile applications design *

Due to modern business demands, IBM joined hands with other companies to develop an open-source business blockchain network called Hyperledger Fabric project that is touching the sky. Due to modular architecture, digital keys, and on-demand data retrieval, hyperledger fabric is regarded as the base for the world’s future modular architecture blockchain-based apps. Further benefits of Hyperledger Fabric are given below to facilitate your interpretation.

sashalisik Sep 6 2019 at 13:27

How we created IoT system for managing solar energy usage

5 min

1.3K

System Analysis and Design *IT Infrastructure *Big Data *Smart House IOT

If you have no idea about the development architecture and mechanical/electrical design behind IoT solutions, they could seem like "having seemingly supernatural qualities or powers". For example, if you show a working IoT system to 18th century people, they'd think it's magic.This article is sort of busting such myth. Or, to put it more technically, about hints for fine-tuning the IoT development for an awesome project in solar energy management area.

XaBoK Mar 17 2021 at 03:35

Architecting Architecture: Makers and Takers

12 min

1.6K

System Analysis and Design *Product Management *Systems engineering *

Translation

The step has been made. Not sure where to, but for sure from the point of no return. Keep calm and keep walking. It is about time to look around and understand the smelly and slippery route before you. And what are those noisy creatures swarming around our fishy “innovative” design we called Mandelbrot blueprint? You don't get a buzzing noise like that, just buzzing and buzzing, without its meaning something.

OBIEESupport Mar 28 2019 at 14:58

How to quickly prepare for a job interview with questions on algorithms and technologies

6 min

РДТЕХ (Разумные Деловые Технологии) corporate blog Entertaining tasks Java *System Analysis and Design *IT career

Greetings to all readers of Habr! My name is Yuriy, I have been teaching high technologies, Oracle, Microsoft and others for more than 20 years, as well as creating, developing and supporting loaded information systems for various business customers. Today I would like to tell you about the current direction: interviews on data processing technologies. The Russian variant of this post you can find here.

It doesn't really make sense for an employer to ask the applicant about traditional programming technologies. That is why I'm going to tell you how to prepare for an interview in only one narrow area related to information processing languages, namely, the processing of long integers(long arithmetic) and the identification of information properties of real world objects, which are described in long integers.