andrewbo29 September 18, 2019 at 05:22 PM

How we made landmark recognition in Cloud Mail.ru, and why

• Mail.ru Group corporate blog,
• Algorithms,
• Image processing,
• Machine learning,
• Artificial Intelligence

With the advent of mobile phones with high-quality cameras, we started making more and more pictures and videos of bright and memorable moments in our lives. Many of us have photo archives that extend back over decades and comprise thousands of pictures which makes them increasingly difficult to navigate through. Just remember how long it took to find a picture of interest just a few years ago.

One of Mail.ru Cloud’s objectives is to provide the handiest means for accessing and searching your own photo and video archives. For this purpose, we at Mail.ru Computer Vision Team have created and implemented systems for smart image processing: search by object, by scene, by face, etc. Another spectacular technology is landmark recognition. Today, I am going to tell you how we made this a reality using Deep Learning.

Read more →

blondnika August 21, 2019 at 01:46 PM

How to Make Emails and Not Mess Up: Practical Tips

• Mail.ru Group corporate blog,
• Spamming and anti-spamming,
• IT systems testing,
• Email layout,
• Web services testing

• Tutorial

A developer, who first encountered generating emails, has almost no chance to write an application, that will do it correctly. Around 40% of emails, generated by corporate applications, are violating some form of standard, and due to this, there are problems with delivery and display. There are reasons for this: emails are technically more difficult than the web, and operating emails is regulated by a few hundred standards, as well as an uncountable number of generally accepted (and not as much) practices, whereas the email clients are more varied and unpredictable than browsers. Testing may significantly improve the situation, but materials that are dedicated to testing the email system, are practically non-existent.

Mail.ru regularly interacts with its users by email. In our projects, all the components responsible for generating emails and even individual mailings, are subject to mandatory testing. In this article, we will share our experience (learning from our mistakes).

Read more →

gen December 18, 2019 at 12:48 PM

Accelerating PHP connectors for Tarantool using Async, Swoole, and Parallel

• Mail.ru Group corporate blog,
• High performance,
• PHP,
• SQL,
• NoSQL

In the PHP ecosystem, there are currently two connectors for the Tarantool server: the official PECL extension tarantool/tarantool-php written in C, and tarantool-php/client written in PHP. I am the author of the latter one.

In this article I would like to share the results of performance testing of both these libraries and show how you can achieve 3x-5x performance improvement (on synthetic tests!) with minimal changes in code.

Read more →

Tarantool Data Grid: Architecture and Features

• Mail.ru Group corporate blog,
• High performance,
• Lua,
• Data storage,
• Tarantool

In 2017, we won the competition for the development of the transaction core for Alfa-Bank's investment business and started working at once. (Vladimir Drynkin, Development Team Lead for Alfa-Bank's Investment Business Transaction Core, spoke about the investment business core at HighLoad++ 2018.) This system was supposed to aggregate transaction data in different formats from various sources, unify the data, save it, and provide access to it.

In the process of development, the system evolved and extended its functions. At some point, we realized that we created something much more than just application software designed for a well-defined scope of tasks: we created a system for building distributed applications with persistent storage. Our experience served as a basis for the new product, Tarantool Data Grid (TDG).

I want to talk about TDG architecture and the solutions that we worked out during the development. I will introduce the basic functions and show how our product could become the basis for building turnkey solutions.

Read more →

sismetanin August 1, 2019 at 01:35 PM

Contextual Emotion Detection in Textual Conversations Using Neural Networks

• Mail.ru Group corporate blog,
• Python,
• Data Mining,
• Big Data,
• Machine learning

Nowadays, talking to conversational agents is becoming a daily routine, and it is crucial for dialogue systems to generate responses as human-like as possible. As one of the main aspects, primary attention should be given to providing emotionally aware responses to users. In this article, we are going to describe the recurrent neural network architecture for emotion detection in textual conversations, that participated in SemEval-2019 Task 3 “EmoContext”, that is, an annual workshop on semantic evaluation. The task objective is to classify emotion (i.e. happy, sad, angry, and others) in a 3-turn conversational data set.

Read more →

vasiliy-t October 21, 2019 at 07:09 PM

Tarantool Kubernetes Operator

• Mail.ru Group corporate blog,
• High performance,
• Database Administration,
• Kubernetes,
• Tarantool

Kubernetes has already become a de-facto standard for running stateless applications, mainly because it can reduce time-to-market for new features. Launching stateful applications, such as databases or stateful microservices, is still a complex task, but companies have to meet the competition and maintain a high delivery rate. So they create a demand for such solutions.

We want to introduce our solution for launching stateful Tarantool Cartridge clusters: Tarantool Kubernetes Operator, more under the cut.

Read more →

sannikovdmitry December 2, 2019 at 07:31 PM

СodeSide. The new game for Russian AI Cup

• Mail.ru Group corporate blog,
• Abnormal programming,
• Sport programming,
• Entertaining tasks,
• Artificial Intelligence

vibe_crc February 20, 2019 at 11:31 AM

Designing Sound for Pathfinder: Kingmaker

• Mail.ru Group corporate blog,
• Game development,
• Sound

voftik April 6, 2019 at 12:36 AM

Liza Alert: volunteers, who save lives

• Mail.ru Group corporate blog,
• Crowdsourcing,
• Multicopters

Liza Alert search-and-rescue team has existed for eight years. It’s a volunteer organization, the fellowship of the ones who care, that searches for missing people effectively collaborating with the Ministry of Emergency Situations and the Ministry of Internal Affairs. Liza Alert collects the entries on missing people; they conduct various educational events, search management and search operations themselves. The team isn’t involved in any business activities, doesn’t have a checking account and doesn’t accept monetary donations.

Mail.Ru has recently provided Liza Alert with a free platform for services, extensively used for search-and-rescue operations. We decided to talk to Sergey Chumak — the head of Liza Alert IT branch — about the work of the volunteer emergency response group and how high-tech solutions aid them.

Read more →

Who is stealing virtual CPU time?

• Mail.ru Group corporate blog,
• Configuring Linux,
• System administration,
• Virtualization

Hi! In this article, I want to explain, in layman’s terms, how steal appears in VMs and tell you about some of the less-than-obvious artifacts that we found during research on the topic that I was involved in as CTO of the Mail.ru Cloud Solutions platform. The platform runs KVM.

Read more →

nikitastupin June 19, 2019 at 03:47 PM

Security of mobile OAuth 2.0

• Mail.ru Group corporate blog,
• Information Security,
• Development of mobile applications

Popularity of mobile applications continues to grow. So does OAuth 2.0 protocol on mobile apps. It's not enough to implement standard as is to make OAuth 2.0 protocol secure there. One needs to consider the specifics of mobile applications and apply some additional security mechanisms.

In this article, I want to share the concepts of mobile OAuth 2.0 attacks and security mechanisms used to prevent such issues. Described concepts are not new but there is a lack of the structured information on this topic. The main aim of the article is to fill this gap.

Read more →

kitashov July 12, 2019 at 11:11 AM

AI-Based Photo Restoration

• Mail.ru Group corporate blog,
• Algorithms,
• Image processing,
• Machine learning

Hi everybody! I’m a research engineer at the Mail.ru Group computer vision team. In this article, I’m going to tell a story of how we’ve created AI-based photo restoration project for old military photos. What is «photo restoration»? It consists of three steps:

• we find all the image defects: fractures, scuffs, holes;
  
• we inpaint the discovered defects, based on the pixel values around them;
  
• we colorize the image.
  

Further, I’ll describe every step of photo restoration and tell you how we got our data, what nets we trained, what we accomplished, and what mistakes we made.

Read more →

pushtaev January 17, 2019 at 09:29 AM

Tips and tricks from my Telegram-channel @pythonetc, December 2018

• Mail.ru Group corporate blog,
• Python,
• Programming

It is new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Read more →

pushtaev December 17, 2019 at 12:57 PM

Tips and tricks from my Telegram-channel @pythonetc, November 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

Tips and tricks from my Telegram-channel @pythonetc, November 2019

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications.

PATH is an environment variable that stores paths where executables are looked for. When you ask your shell to run ls, the shell looks for the ls executable file across all paths that are presented in PATH.

Read more →

dokshina December 16, 2019 at 12:41 PM

Deploying Tarantool Cartridge applications with zero effort (Part 1)

• Mail.ru Group corporate blog,
• High performance,
• IT Infrastructure,
• Database Administration,
• Distributed systems

We have already presented Tarantool Cartridge that allows you to develop and pack distributed applications. Now let's learn how to deploy and control these applications. No panic, it's all under control! We have brought together all the best practices of working with Tarantool Cartridge and wrote an Ansible role, which will deploy the package to servers, start and join instances into replica sets, configure authorization, bootstrap vshard, enable automatic failover and patch cluster configuration.

Interesting, huh? Dive in, check details under the cut.

Read more →

pushtaev June 4, 2019 at 04:21 PM

Tips and tricks from my Telegram-channel @pythonetc, May 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Read more →

lenkis January 28, 2019 at 04:46 PM

Lua in Moscow 2019 conference

• Mail.ru Group corporate blog,
• Lua,
• Conferences

On the first Sunday of March, Mail.ru Group’s Moscow office will be hosting the third international Lua conference, Lua in Moscow 2019. The program features talks by Roberto Ierusalimschy and the leading experts in Lua and LuaJIT from Russia and other countries.

Lua is a unique programming language used not only in computer games, but also as an embedded language in such web-programming products as Redis, nginx, Tarantool, OpenResty. Lua is also used for big data analysis and scientific calculations. You can find Lua in many routers, printers and other devices.

You are welcome to join, even if you haven’t been writing in Lua so far. We bet the conference will give you unexpected insights!

Read more →

Rosik May 12, 2020 at 06:08 PM

The Anatomy of LuaJIT Tables and What’s Special About Them

• Mail.ru Group corporate blog,
• Programming,
• Perfect code,
• Designing and refactoring,
• Lua

I don't know about you, but I really like to get inside all sorts of systems. In this article, I’m going to tell you about the internals of Lua tables and special considerations for their use. Lua is my primary professional programming language, and if one wants to write good code, one needs at least to peek behind the curtain. If you are curious, follow me.

Read more →

sismetanin April 30, 2019 at 11:42 AM

Google News and Leo Tolstoy: visualizing Word2Vec word embeddings using t-SNE

• Mail.ru Group corporate blog,
• Python,
• Big Data,
• Data visualization,
• Machine learning

Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

Read more →

pushtaev November 13, 2019 at 04:31 PM

Tips and tricks from my Telegram-channel @pythonetc, October 2019

• Mail.ru Group corporate blog,
• Python,
• Programming

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

If you want to iterate over several iterables at once, you can use the zip function (it has nothing to do with ZIP file format):

Read more →