Mail.ru Group

andrewbo29 September 18, 2019 at 05:22 PM

How we made landmark recognition in Cloud Mail.ru, and why

• Mail.ru Group corporate blog,
• Algorithms,
• Image processing,
• Machine learning,
• Artificial Intelligence

With the advent of mobile phones with high-quality cameras, we started making more and more pictures and videos of bright and memorable moments in our lives. Many of us have photo archives that extend back over decades and comprise thousands of pictures which makes them increasingly difficult to navigate through. Just remember how long it took to find a picture of interest just a few years ago.

One of Mail.ru Cloud’s objectives is to provide the handiest means for accessing and searching your own photo and video archives. For this purpose, we at Mail.ru Computer Vision Team have created and implemented systems for smart image processing: search by object, by scene, by face, etc. Another spectacular technology is landmark recognition. Today, I am going to tell you how we made this a reality using Deep Learning.

Read more →

blondnika August 21, 2019 at 01:46 PM

How to Make Emails and Not Mess Up: Practical Tips

• Mail.ru Group corporate blog,
• Spamming and anti-spamming,
• IT systems testing,
• Email layout,
• Web services testing

• Tutorial

A developer, who first encountered generating emails, has almost no chance to write an application, that will do it correctly. Around 40% of emails, generated by corporate applications, are violating some form of standard, and due to this, there are problems with delivery and display. There are reasons for this: emails are technically more difficult than the web, and operating emails is regulated by a few hundred standards, as well as an uncountable number of generally accepted (and not as much) practices, whereas the email clients are more varied and unpredictable than browsers. Testing may significantly improve the situation, but materials that are dedicated to testing the email system, are practically non-existent.

Mail.ru regularly interacts with its users by email. In our projects, all the components responsible for generating emails and even individual mailings, are subject to mandatory testing. In this article, we will share our experience (learning from our mistakes).

Read more →

Tarantool Data Grid: Architecture and Features

• Mail.ru Group corporate blog,
• High performance,
• Lua,
• Data storage,
• Distributed systems

In 2017, we won the competition for the development of the transaction core for Alfa-Bank's investment business and started working at once. (Vladimir Drynkin, Development Team Lead for Alfa-Bank's Investment Business Transaction Core, spoke about the investment business core at HighLoad++ 2018.) This system was supposed to aggregate transaction data in different formats from various sources, unify the data, save it, and provide access to it.

In the process of development, the system evolved and extended its functions. At some point, we realized that we created something much more than just application software designed for a well-defined scope of tasks: we created a system for building distributed applications with persistent storage. Our experience served as a basis for the new product, Tarantool Data Grid (TDG).

I want to talk about TDG architecture and the solutions that we worked out during the development. I will introduce the basic functions and show how our product could become the basis for building turnkey solutions.

Read more →

sismetanin August 1, 2019 at 01:35 PM

Contextual Emotion Detection in Textual Conversations Using Neural Networks

• Mail.ru Group corporate blog,
• Python,
• Data Mining,
• Big Data,
• Machine learning

Nowadays, talking to conversational agents is becoming a daily routine, and it is crucial for dialogue systems to generate responses as human-like as possible. As one of the main aspects, primary attention should be given to providing emotionally aware responses to users. In this article, we are going to describe the recurrent neural network architecture for emotion detection in textual conversations, that participated in SemEval-2019 Task 3 “EmoContext”, that is, an annual workshop on semantic evaluation. The task objective is to classify emotion (i.e. happy, sad, angry, and others) in a 3-turn conversational data set.

Read more →

Tarantool Kubernetes Operator

• Mail.ru Group corporate blog,
• High performance,
• IT Infrastructure,
• Database Administration,
• Kubernetes

Kubernetes has already become a de-facto standard for running stateless applications, mainly because it can reduce time-to-market for new features. Launching stateful applications, such as databases or stateful microservices, is still a complex task, but companies have to meet the competition and maintain a high delivery rate. So they create a demand for such solutions.

We want to introduce our solution for launching stateful Tarantool Cartridge clusters: Tarantool Kubernetes Operator, more under the cut.

Read more →

kitashov July 12, 2019 at 11:11 AM

AI-Based Photo Restoration

• Mail.ru Group corporate blog,
• Algorithms,
• Image processing,
• Machine learning

Hi everybody! I’m a research engineer at the Mail.ru Group computer vision team. In this article, I’m going to tell a story of how we’ve created AI-based photo restoration project for old military photos. What is «photo restoration»? It consists of three steps:

• we find all the image defects: fractures, scuffs, holes;
  
• we inpaint the discovered defects, based on the pixel values around them;
  
• we colorize the image.
  

Further, I’ll describe every step of photo restoration and tell you how we got our data, what nets we trained, what we accomplished, and what mistakes we made.

Read more →

nikitastupin June 19, 2019 at 03:47 PM

Security of mobile OAuth 2.0

• Mail.ru Group corporate blog,
• Information Security,
• Development of mobile applications

Popularity of mobile applications continues to grow. So does OAuth 2.0 protocol on mobile apps. It's not enough to implement standard as is to make OAuth 2.0 protocol secure there. One needs to consider the specifics of mobile applications and apply some additional security mechanisms.

In this article, I want to share the concepts of mobile OAuth 2.0 attacks and security mechanisms used to prevent such issues. Described concepts are not new but there is a lack of the structured information on this topic. The main aim of the article is to fill this gap.

Read more →

Who is stealing virtual CPU time?

• Mail.ru Group corporate blog,
• Configuring Linux,
• System administration,
• Virtualization

Hi! In this article, I want to explain, in layman’s terms, how steal appears in VMs and tell you about some of the less-than-obvious artifacts that we found during research on the topic that I was involved in as CTO of the Mail.ru Cloud Solutions platform. The platform runs KVM.

Read more →

voftik April 6, 2019 at 12:36 AM

Liza Alert: volunteers, who save lives

• Mail.ru Group corporate blog,
• Crowdsourcing,
• Multicopters

Liza Alert search-and-rescue team has existed for eight years. It’s a volunteer organization, the fellowship of the ones who care, that searches for missing people effectively collaborating with the Ministry of Emergency Situations and the Ministry of Internal Affairs. Liza Alert collects the entries on missing people; they conduct various educational events, search management and search operations themselves. The team isn’t involved in any business activities, doesn’t have a checking account and doesn’t accept monetary donations.

Mail.Ru has recently provided Liza Alert with a free platform for services, extensively used for search-and-rescue operations. We decided to talk to Sergey Chumak — the head of Liza Alert IT branch — about the work of the volunteer emergency response group and how high-tech solutions aid them.

Read more →

vibe_crc February 20, 2019 at 11:31 AM

Designing Sound for Pathfinder: Kingmaker

• Mail.ru Group corporate blog,
• Game development,
• Sound

pushtaev January 17, 2019 at 09:29 AM

Tips and tricks from my Telegram-channel @pythonetc, December 2018

• Mail.ru Group corporate blog,
• Python,
• Programming

It is new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Read more →

pushtaev June 4, 2019 at 04:21 PM

Tips and tricks from my Telegram-channel @pythonetc, May 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Read more →

lenkis January 28, 2019 at 04:46 PM

Lua in Moscow 2019 conference

• Mail.ru Group corporate blog,
• Lua,
• Conferences

On the first Sunday of March, Mail.ru Group’s Moscow office will be hosting the third international Lua conference, Lua in Moscow 2019. The program features talks by Roberto Ierusalimschy and the leading experts in Lua and LuaJIT from Russia and other countries.

Lua is a unique programming language used not only in computer games, but also as an embedded language in such web-programming products as Redis, nginx, Tarantool, OpenResty. Lua is also used for big data analysis and scientific calculations. You can find Lua in many routers, printers and other devices.

You are welcome to join, even if you haven’t been writing in Lua so far. We bet the conference will give you unexpected insights!

Read more →

sismetanin April 30, 2019 at 11:42 AM

Google News and Leo Tolstoy: visualizing Word2Vec word embeddings using t-SNE

• Mail.ru Group corporate blog,
• Python,
• Big Data,
• Data visualization,
• Machine learning

Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

Read more →

pushtaev November 13, 2019 at 04:31 PM

Tips and tricks from my Telegram-channel @pythonetc, October 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

If you want to iterate over several iterables at once, you can use the zip function (it has nothing to do with ZIP file format):

Read more →

pushtaev February 4, 2019 at 04:28 PM

Tips and tricks from my Telegram-channel @pythonetc, January 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Previous publications:

• December 2018

Read more →

pushtaev September 5, 2019 at 04:18 PM

Tips and tricks from my Telegram-channel @pythonetc, August 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

If an instance of a class doesn’t have an attribute with the given name, it tries to access the class attribute with the same name.

>>> class A:
...     x = 2
...
>>> A.x
2
>>> A().x
2

Read more →

sismetanin May 16, 2019 at 12:53 PM

In-App Updates Flexible Flow: Speed Up the App Update Process on Android

• Mail.ru Group corporate blog,
• Programming,
• Development of mobile applications,
• Development for Android

With a variety of new tools and features announced at Android Dev Summit, special attention should be given to the In-App Updates (IAUs) API allowing developers to increase the speed of delivering features, bug-fixes and performance improvements to active users. Since this feature was finally released after Google I/O 2019, in this article I’ll deep dive on IAUs API, describe in details recommended user flows and provide with some code samples. Moreover, I'll share some experience of IAUs integration in the Pandao app, a marketplace platform for Chinese goods.

Read more →

Rosik October 9, 2019 at 05:32 PM

Тarantool Cartridge: Sharding Lua Backend in Three Lines

• Mail.ru Group corporate blog,
• High performance,
• Server optimization,
• Lua,
• Distributed systems

In Mail.ru Group, we have Tarantool, a Lua-based application server and a database united. It's fast and classy, but the resources of a single server are always limited. Vertical scaling is also not the panacea. That is why Tarantool has some tools for horizontal scaling, or the vshard module [1]. It allows you to spread data across multiple servers, but you'll have to tinker with it for a while to configure it and bolt on the business logic.

Good news: we got our share of bumps (for example, [2], [3]) and created another framework, which significantly simplifies the solution to this problem.

Тarantool Cartridge is the new framework for developing complex distributed systems. It allows you to concentrate on writing business logic instead of solving infrastructure problems. Under the cut, I will tell you how this framework works and how it could help in writing distributed services.

Read more →

pushtaev October 8, 2019 at 11:10 AM

Tips and tricks from my Telegram-channel @pythonetc, September 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Read more →