Pull to refresh
397.9

Python *

Interpreted high-level programming language for general-purpose programming

Show first
Rating limit
Level of difficulty

Implementing Offline traceroute Tool Using Python

Reading time30 min
Views4.3K

Hey everyone! This post was born from a question asked by an IT forum member. The summary of the question looked as follows:


  • There is a set of text files containing routing tables collected from various network devices.
  • Each file represents one device.
  • Device platforms and routing table formats may vary.
  • It is required to analyze a routing path from any device to an arbitrary subnet or host on-demand.
  • Resulting output should contain a list of routing table entries that are used for the routing to the given destination on each hop.

The one who asked a question worked as a TAC engineer. It is often that they collect or receive from the customers some text 'snapshots' of the network state for further offline analysis while troubleshooting the issues. Some automation could really save a lot of time.


I found this task interesting and also applicable to my own needs, so I decided to write a Proof-of-Concept implementation in Python 3 for Cisco IOS, IOS-XE, and ASA routing table format.


In this article, I’ll try to reconstruct the resulting script development process and my considerations behind each step.


Let’s get started.

Read more →

Coins Classification using Neural Networks

Reading time19 min
Views3.1K

See more at robotics.snowcron.comThis is the first article in a serie dedicated to coins classification.Having countless "dogs vs cats" or "find a pedestrian on the street" classifiers all over the Internet, coins classification doesn't look like a difficult task. At first. Unfortunately, it is degree of magnitude harder - a formidable challenge indeed. You can easily tell heads of tails? Great. Can you figure out if the number is 1 mm shifted to the left? See, from classifier's view it is still the same head... while it can make a difference between a common coin priced according to the number on it and a rare one, 1000 times more expensive.Of course, we can do what we usually do in image classification: provide 10,000 sample images... No, wait, we can not. Some types of coins are rare indeed - you need to sort through a BASKET (10 liters) of coins to find one. Easy arithmetics suggests that to get 10000 images of DIFFERENT coins you will need 10,000 baskets of coins to start with. Well, and unlimited time.So it is not that easy.Anyway, we are going to begin with getting large number of images and work from there. We will use Russian coins as an example, as Russia had money reform in 1994 and so the number of coins one can expect to find in the pocket is limited. Unlike USA with its 200 years of monetary history. And yes, we are ONLY going to focus on current coins: the ultimate goal of our work is to write a program for smartphone to classify coins you have received in a grocery store as a change.Which makes things even worse, as we can not count on good lighting and quality cameras anymore. But we'll still try.In addition to "only Russian coins, beginning from 1994", we are going to add an extra limitation: no special occasion coins. Those coins look distinctive, so anyone can figure that this coin is special. We focus on REGULAR coins. Which limits their number severely.Don't take me wrong: if we need to apply the same approach to a full list of coins... it will work. But I got 15 GB of images for that limited set, can you imagine how large the complete set will be?!To get images, I am going to scan one of the largest Russian coins site "meshok.ru".This site allows buyers and sellers to find each other; sellers can upload images... just what we need. Unfortunately, a business-oriented seller can easily upload his 1 rouble image to 1, 2, 5, 10 roubles topics, just to increase the exposure.

So we can not count on the topic name, we have to determine what coin is on the photo ourselves.To scan the site, a simple scanner was written, based on the Python's Beautiful Soup library. In just few hours I got over 50,000 photos. Not a lot by Machine Learning standards, but definitely a start.After we got the images, we have to - unfortunately - revisit them by hand, looking for images we do not want in our training set, or for images that should be edited somehow. For example, someone could have uploaded a photo of his cat. We don't need a cat in our dataset.First, we delete all images, that can not be split to head/hail.

Читать далее

Doing «Data Science» even if you have never heard the words before

Reading time12 min
Views1.4K

There’s a lot of talk about machine learning nowadays. A big topic – but, for a lot of people, covered by this terrible layer of mystery. Like black magic – the chosen ones’ art, above the mere mortal for sure. One keeps hearing the words “numpy”, “pandas”, “scikit-learn” - and looking each up produces an equivalent of a three-tome work in documentation.

I’d like to shatter some of this mystery today. Let’s do some machine learning, find some patterns in our data – perhaps even make some predictions. With good old Python only – no 2-gigabyte library, and no arcane knowledge needed beforehand.

Interested? Come join us.

Read more

Visualizing Network Topologies: Zero to Hero in Two Days

Reading time49 min
Views33K

Hey everyone! This is a follow-up article on a local Cisco Russia DevNet Marathon online event I attended in May 2020. It was a series of educational webinars on network automation followed by daily challenges based on the discussed topics.
On a final day, the participants were challenged to automate a topology analysis and visualization of an arbitrary network segment and, optionally, track and visualize the changes.


The task was definitely not trivial and not widely covered in public blog posts. In this article, I would like to break down my own solution that finally took first place and describe the selected toolset and considerations.

Let's get started.


Read more →

Big Data Tools EAP 12 Is Out: Experimental Python Support and Search Function in Zeppelin Notebooks

Reading time3 min
Views1.2K

Update 12 of the Big Data Tools plugin for IntelliJ IDEA Ultimate, PyCharm Professional Edition, and DataGrip has been released. You can install it from the JetBrains Plugin Repository or from inside your IDE. The plugin allows you to edit Zeppelin notebooks, upload files to cloud filesystems, and monitor Hadoop and Spark clusters.


In this release, we've added experimental Python support and global search inside Zeppelin notebooks. We’ve also addressed a variety of bugs. Let's talk about the details.


Read more →

Toxic Comments Detection in Russian

Reading time17 min
Views7.9K

Currently, social network sites tend to be one of the major communication platforms in both offline and online space. Freedom of expression of various points of view, including toxic, aggressive, and abusive comments, might have a long-term negative impact on people’s opinions and social cohesion. As a consequence, the ability to automatically identify and moderate toxic content on the Internet to eliminate the negative consequences is one of the necessary tasks for modern society. This paper aims at the automatic detection of toxic comments in the Russian language. As a source of data, we utilized anonymously published Kaggle dataset and additionally validated its annotation quality. To build a classification model, we performed fine-tuning of two versions of Multilingual Universal Sentence Encoder, Bidirectional Encoder Representations from Transformers, and ruBERT. Finetuned ruBERT achieved F1 = 92.20%, demonstrating the best classification score. We made trained models and code samples publicly available to the research community.
Read more →

Making python's dream of multithreading come true

Reading time10 min
Views6.5K

Intro


So you are writing some CPU-intensive code in Python and really trying to find ways out of its single-threaded prison. You might be looking towards Numba's "nopython parallel" mode, you might be using forked processes with multiprocessing, you might be writing microservices with database-like coordinators, or even writing your own multithreaded programs in C/C++ just like creators of TensorFlow did.


In this article I'm describing a rationale for my pet project where I try to implement facilities for general purpose multitasking to be used in a form of simple python code, employing a database-like approach for interpreters communication, while keeping the GIL (Global Interpreter Lock) and trying to be as pythonic as possible.



It could also become handy in the light of upcoming multiple interpreters support in CPython.


As far as I know, nobody came that far in trying to provide Python program with native shareable storage. The last closest attempt was Python Object Sharing which is pretty much dead by now. I hope my project won't meet the same fate.

Read more →

Leading Full Stack Development Companies to Check Out in 2020

Reading time4 min
Views2.4K
Given the diverse range of responsibilities and technologies out there today, it is no wonder companies choose to employ full stack development services to achieve end goals without hiring multiple people and struggling to attain streamlined workflows.

What is full-stack development?


Full-stack development services refer to the development of both client-side and server-side interfaces of any application. In technical terms, these are the front-end and back-end, respectively.

A client-side-only developer typically is responsible for what users see and can interact within an application or a website. They use a specific set of languages, including HTML and CSS. They also use particular frameworks and libraries that include AngularJS, Bootstrap and ReactJS. On the other hand, a back-end developer typically focuses on how the website works and is built. The primary languages are Python, JavaScript and C++ among others.

To hire a full-stack web developer is to get someone who is well-versed in both front-end and back-end development and has the skills and technological prowess to step in at any stage of a project.

What qualities does the ideal full-stack developer possess?


If you’re looking to hire a remote full-stack web developer or a full-time one, consider looking out for these qualities:

  • Mastery over front-end technologies
  • Knowledge of at least one server-side programming language
  • Concrete understanding of DBMS technology
  • UI/UX design skills
  • Experience in handling servers
  • Knowledge of Git and version control systems
  • In-depth understanding of web services or API
  • Awareness of security concerns and frameworks
  • Understanding of algorithms and data structure
Read more →

Machine learning in browser: ways to cook up a model

Reading time12 min
Views2.2K

With ML projects still on the rise we are yet to see integrated solutions in almost every device around us. The need for processing power, memory and experimentation has led to machine learning and DL frameworks targeting desktop computers first. However once trained, a model may be executed in a more constrained environment on a smartphone or on an IoT device. A particularly interesting environment to run the model on is browser. Browser-based solutions may be used on a wide range of devices, desktop and mobile, online and offline. The topic of this post is how to prepare a model for the in-browser usage.

This post presents an end-to-end implementations of a model creation in Python and Node.js. The end goal is to create a model and to use it in a browser. I'll use TensorFlow and TensorFlow.js as main frameworks. One could train a model in Python and convert it to JS. Alternative is to train a model directly in javascript, hence omitting the conversion step.

I have more experience in Python and use it in my everyday work. I occasionally use javascript, but have very little experience in the contemporary front-end development. My hope from this post that python developers with little JS experience could use it to kick start their JS usage.

Read more

Crime, Race and Lethal Force in the USA — Part 3

Reading time24 min
Views1.7K
image
This is the concluding part of my article devoted to a statistical analysis of police shootings and criminality among the white and the black population of the United States. In the first part, we talked about the research background, goals, assumptions, and source data; in the second part, we investigated the national use-of-force and crime data and tracked their connection with race.
Read more →

Crime, Race and Lethal Force in the USA — Part 1

Reading time8 min
Views2.6K
image

Do the police in the US really shoot black people more often than white people? Is use of lethal force connected with race? How is crime related to race? What are the odds of getting shot by the police if you are white and if you are black? We're taking public data and python with pandas to shed some light on these questions, propaganda and politics set far aside.
Read more →

7 Ultimate Programming languages For Mobile App Development

Reading time5 min
Views4.6K
"image"

Do you know that 21% of people open an app 50+ times per day? Yes, you heard that right. With the rapid development of technology, the mobile app now made many things possible, which was previously unthinkable.

And that's why there is an incredible increase in the number of mobile users. According to a recent mobile app development stat, the number of mobile users worldwide is projected to increase to 6.95 billion by the end of 2020.

In the last couple of years, the mobile app development industry has grown manifold, changing how businesses function around the world. If you are planning to jump into mobile app development, then choosing the right programming language will be the most significant challenge.

There are more than 600 programming languages, and each one has its own perks and popularity. Are you pondering which language would be best for developing a stunning app?

Several factors come to mind when making this choice, but the most important one is the language's demand. Here in this blog, I have listed the best programming language for mobile apps in terms of popularity and demand. Let's start!
Read more →

Machine Learning CPython library 'VKF'

Reading time14 min
Views1.4K
Previous article describes a web server for Machine Learning system 'VKF' based on Lattice Theory. This paper is an attempt to explain details of using the CPython library directly. We reproduce working sessions of experiments on datasets 'Mushroom' and 'Wine Quality' from UCI Machine Learning repository. The structures of input files are discussed too.


Read more →

How to find an English teacher. Part 2

Reading time4 min
Views910
image

This is a continuation of story about using Data Science for finding an English teacher. If you have not read it yet - there is an opportunity to become familiar with it

Briefly  -  we had information about language teachers and tried to apply some basic ideas using pandas and our expectations. Unfortunately we got stuck on the third step, because there is not enough information for resolving our the last requirements  -  we need not more 3 candidates at the end.

Disclaimer
It is an approach based on my own experience and can be unsuitable to your point of view, ideas, or principles.

Web server for Machine Learning 'VKF-solver'

Reading time20 min
Views1.6K
Nowadays most people identify Machine Learning with training of various kinds of neural networks. At the beginning there were fully connected networks, then convolutional and recurrent networks replace them, now there exist a quite exotic variants of networks such that GAN and LTSM networks.

Their training requires constantly increasing volume of samples, and they also do not be able to explain why a particular decision was made. Structural approaches to Machine Learning avoiding these drawbacks exist, the software implementation of one of which is described in the article. This is an English translation of original post by the author.


Read more →

How to find an English teacher. Part 1

Reading time5 min
Views1.6K


In the modern world, here and there ideas are arising about using data science for an extra benefit. For instance, Google can use a history of watched videos for providing recommendations about new ones. Online shops are using a recommendation system for increasing your receipt. However… if companies use the data for their benefit, could we do the same for own needs such as looking an online English teacher?


Disclaimer

It is an approach based on my own experience and can be unsuitable to your point of view, ideas, or principles.

Approach to calculating individual risk in COVID-19

Reading time3 min
Views1.2K

In February 2020, when the disease came to Europe, it became apparent to me that our timid hopes that the epidemics would subside and be finally buried in the China's soil were ruined. It was already evident from the Chinese statistics that the virus is lethal enough to scare and mild enough to pass unnoticed in many cases and, thus, to guarantee its effective dissemination. The question was when it reaches each next country.




Another question was the individual risks, especially the risk of lethal outcome if one contracts the virus. The average figure of around 5% was circulated by late January and early February. It was known that males were more susceptible to fatal outcomes. By February, it was also evident that the virus doesn't lead to death only in the elderly — the middle age was significantly affected, as well.

Read more →

Is Python The Answer To High-End Mobile Apps?

Reading time4 min
Views2.8K
image

Before the development of any software or a mobile app, the first thing that comes in mind is- which programming language is the best for this app. Yes! It is really important to choose the best programming language for particular mobile app development.

So, among various programming languages, let’s throw light on the significance of Python in Mobile app development or it could also be said that why Python is the best language for mobile app development?

Before that, let’s have a look at the stat where according to statista.com, Python remains the most popular programming language in 2020, according to GitHub and Google Trends, surpassing longstanding Java and JavaScript in popularity.

Python was also the biggest gainer in the Tiobe index of the popularity of programming languages in the year 2018, rising by 3.62% points from January 2018 to January 2019, with a rating of approx. 8.2% for this month.
Read more →

Authors' contribution