Some time ago, in a discussion on one of SObjectizer's releases, we were asked: "Is it possible to make a DSL to describe a data-processing pipeline?" In other words, is it possible to write something like that:

A | B | C | D

and get a working pipeline where messages are going from A to B, and then to C, and then to D. With control that B receives exactly that type that A returns. And C receives exactly that type that B returns. And so on.

It was an interesting task with a surprisingly simple solution. For example, that's how the creation of a pipeline can look like:

auto pipeline = make_pipeline(env, stage(A) | stage(B) | stage(C) | stage(D));

Or, in a more complex case (that will be discussed below):

auto pipeline = make_pipeline( sobj.environment(),
        stage(validation) | stage(conversion) | broadcast(
            stage(archiving),
            stage(distribution),
            stage(range_checking) | stage(alarm_detector{}) | broadcast(
                stage(alarm_initiator),
                stage( []( const alarm_detected & v ) {
                        alarm_distribution( cerr, v );
                    } )
                )
            ) );

In this article, we'll speak about the implementation of such pipeline DSL. We'll discuss mostly parts related to stage(), broadcast() and operator|() functions with several examples of usage of C++ templates. So I hope it will be interesting even for readers who don't know about SObjectizer (if you never heard of SObjectizer here is an overview of this tool).

Roslyn, the .NET compiler platform, helps you catch bugs even before you run your code. One example is Roslyn’s spellcheck analyzer that is built into Visual Studio. Let’s say you are creating a static method and misspelled the word static as statc. You will be able to see this spelling error before you run your code because Roslyn can produce warnings in your code as you type even before you’ve finished the line. In other words, you don’t have to build your code to find out that you made a mistake.



Roslyn analyzers can also surface an automatic code fix through the Visual Studio light bulb icon that allows you to fix your code immediately.

Some time ago, the creator of Lua programming language, Roberto Ierusalimschy, visited our Moscow office. We asked him some questions that we prepared with the participation of Habr.com users also. And finally, we’d like to share full-text version of this interview.

C++20 adds a new operator, affectionately dubbed the «spaceship» operator: <=>. There was a post awhile back by our very own Simon Brand detailing some information regarding this new operator along with some conceptual information about what it is and does. The goal of this post is to explore some concrete applications of this strange new operator and its associated counterpart, the operator== (yes it has been changed, for the better!), all while providing some guidelines for its use in everyday code.

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications


The \ symbol in regular string have special meaning. \t is tab character, \r is carriage return and so on.

You can use raw-strings to disable this behaviour. r'\t' is just backslash and t.

You obviously can’t use ' inside r'...'. However, it still can be escaped by \, but \ is preserved in the string:

Beginning with C# 8.0 on .NET Core 3.0, you can define an implementation when you declare a member of an interface. The most common scenario is to safely add members to an interface already released and used by innumerable clients.

In this tutorial, you'll learn how to:

• Extend interfaces safely by adding methods with implementations.
• Create parameterized implementations to provide greater flexibility.
• Enable implementers to provide a more specific implementation in the form of an override.

I don’t care what your dragon’s said, it’s a lie. Dragons lie. You don’t know what’s waiting for you on the other side.

Michael Swanwick, The Iron Dragon’s Daughter

This article is based on the post in the Krister Walfridsson’s blog, “Why undefined behavior may call a never called function?”.

The article draws a simple conclusion: undefined behavior in a compiler can do anything, even something absolutely unexpected. In this article, I examine the internal mechanism of this optimization works.

We are pleased to announce that the June 2019 release of the Python Extension for Visual Studio Code is now available. You can download the Python extension from the Marketplace, or install it directly from the extension gallery in Visual Studio Code. If you already have the Python extension installed, you can also get the latest update by restarting Visual Studio Code. You can learn more about Python support in Visual Studio Code in the documentation.

In this release we made improvements that are listed in our changelog, closing a total of 70 issues including a plot viewer with the Python Interactive window, parallel tests with pytest, and indentation of run selection in the terminal.

Every development environment has a tool called «Output». There is no need to describe what it does, since all developers without exception use it in their work on a daily basis. It is simple and conservative. 

It has remained essentially unchanged for decades, and to this day looks something like this:


Text, text, and more text. Lots of text...

Even in this tiny example the line containing the error is not immediately apparent. Finding it takes time and effort. Simply because one has to read through the text and search for the words «error», «exception» or «warning». The programmer has to search, and the client has to pay for the time spent searching.

A few words about SObjectizer and its history

SObjectizer is a rather small C++ framework that simplifies the development of multithreaded applications. SObjectizer allows a developer to use approaches from Actor, Publish-Subscribe and Communicating Sequential Processes (CSP) models. It's an OpenSource project that is distributed under BSD-3-CLAUSE license.

SObjectizer has a long history. SObjectizer itself was born in 2002 as SObjectizer-4 project. But it was based on ideas from previous SCADA Objectizer that was developed between 1995 and 2000. SObjectizer-4 was open-sourced in 2006, but its evolution was stopped soon after that. A new version of SObjectizer with the name SObjectizer-5 was started in 2010 and was open-sourced in 2013. The evolution of SObjectizer-5 is still in progress and SObjectizer-5 has incorporated many new features since 2013.

SObjectizer is more or less known in the Russian segment of the Internet, but almost unknown outside of the exUSSR. It's because the SObjectizer was mainly used for local projects in exUSSR-countries and many articles, presentations, and talks about SObjectizer are in Russian.

A niche for SObjectizer and similar tools

Multithreading is used in Parallel computing as well as in Concurrent computing. But there is a big difference between Parallel and Concurrent computing. And, as a consequence, there are tools targeted Parallel computing, and there are tools for Concurrent computing, and they are different.

Dr. Martin Kleppmann is a researcher in distributed systems at the University of Cambridge, and the author of the highly acclaimed «Designing Data-Intensive Applications» (O'Reilly Media, 2017). 

Kevin Scott, CTO at Microsoft once said: «This book should be required reading for software engineers. Designing Data-Intensive Applications is a rare resource that connects theory and practice to help developers make smart decisions as they design and implement data infrastructure and systems.»

Martin’s main research interests include collaboration software, CRDTs, and formal verification of distributed algorithms. Previously he was a software engineer and an entrepreneur at several Internet companies including LinkedIn and Rapportive, where he worked on large-scale data infrastructure.

Vadim Tsesko (@incubos) is a lead software engineer at Odnoklassniki who works in Core Platform team. Vadim’s scientific and engineering interests include distributed systems, data warehouses and verification of software systems.

Contents:

• Moving from business to academic research;
  
• Discussion of «Designing Data-Intensive Applications»;
  
• Common sense against artificial hype and aggressive marketing;
  
• Pitfalls of CAP theorem and other industry mistakes;
  
• Benefits of decentralization;
  
• Blockchains, Dat, IPFS, Filecoin, WebRTC;
  
• New CRDTs. Formal verification with Isabelle;
  
• Event sourcing. Low level approach. XA transactions; 
  
• Apache Kafka, PostgreSQL, Memcached, Redis, Elasticsearch;
  
• How to apply all that tools to real life;
  
• Expected target audience of Martin’s talks and the Hydra conference.
  

Whether you are new or have been using Visual Studio for years, there are a bunch of tips and tricks that can make you more productive. We’ve been sharing tips on Twitter using the hashtag #vstip for a while, and this is a collection of the best ones so far.

Debugger

Hitting F10 to build, run, and attach debugger instead of F5 will automatically break on the first time your own code is being executed. No breakpoints needed.



Supported from Visual Studio 2005

Reattach to process (Shift+Alt+P) is extremely helpful when you have to attach to the same process again and again.



Supported from Visual Studio 2017 v15.8

A blue dot in the margin indicates a switch of threads while stepping through debugging.



Supported from Visual Studio 2013

Avalonia ui is a delightful framework to which you want to return again and again. So let's go back to it again and look at some of the features with my message box.

Every now and then we create our own problems with our own hands… with our vision of the world… with our inaction… with our laziness… and with our fears. As a result, it seems to become very convenient to swim in the public flow of sewage patterns… because it is warm and fun, and the rest does not matter – we can smell round. But after a fail comes the realization of the simple truth – instead of generating an endless stream of causes, self-pity and self-justification, it is enough just to do what you consider the most important for yourself. This will be the starting point for your new reality.

For me, the written below is just such a starting point. The way is expected to be lingering…

Hello, dear Habr users! This article is about Web 3.0 — the decentralized Internet. Web 3.0 introduces the concept of decentralization as the foundation of the modern Internet. Many computer systems and networks require security and decentralization features to meet their needs. A distributed registry using blockchain technology provides efficient solutions for decentralization.

Some days ago the Windows team announced the May 2019 Update for Windows 10. In this post we’re going to look at what we, the Python team, have done to make Python easier to install on Windows by helping the community publish to the Microsoft Store and, in collaboration with Windows, adding a default “python.exe” command to help find it. You may have already heard about these on the Python Bytes podcast, at PyCon US, or through Twitter.

I guess one of the most important issues in this topic is building an exception handling architecture in your application. This is interesting for many reasons. And the main reason, I think, is an apparent simplicity, which you don’t always know what to do with. All the basic constructs such as IEnumerable, IDisposable, IObservable, etc. have this property and use it everywhere. On the one hand, their simplicity tempts to use these constructs in different situations. On the other hand, they are full of traps which you might not get out. It is possible that looking at the amount of information we will cover you’ve got a question: what is so special about exceptional situations?

However, to make conclusions about building the architecture of exception classes we should learn some details about their classification. Because before building a system of types that would be clear for the user of code, a programmer should determine when to choose the type of error and when to catch or skip exceptions. So, let’s classify the exceptional situations (not the types of exceptions) based on various features.

Specifics of QueryProvider

QueryProvider can’t deal with this:

var result = _context.Humans
                      .Select(x => $"Name: {x.Name}  Age: {x.Age}")
                      .Where(x => x != "")
                      .ToList();

It can’t deal with any sentence using an interpolated string, but it’ll easily deal with this:

var result = _context.Humans
                      .Select(x => "Name " +  x.Name + " Age " + x.Age)
                      .Where(x => x != "")
                      .ToList();

The most painful thing is to fix bugs after turning on ClientEvaluation (exception for client-side calculation), since all Automapper profiles should be strictly analyzed for interpolation. Let’s find out what’s what and propose our solution to the problem.

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Introduction

It’s time to talk about exceptions or, rather, exceptional situations. Before we start, let’s look at the definition. What is an exceptional situation?

This is a situation that makes the execution of current or subsequent code incorrect. I mean different from how it was designed or intended. Such a situation compromises the integrity of an application or its part, e.g. an object. It brings the application into an extraordinary or exceptional state.

But why do we need to define this terminology? Because it will keep us in some boundaries. If we don’t follow the terminology, we can get too far from a designed concept which may result in many ambiguous situations. Let’s see some practical examples:

 struct Number
 {
     public static Number Parse(string source)
     {
         // ...
         if(!parsed)
         {
             throw new ParsingException();
         }
         // ...
     }

     public static bool TryParse(string source, out Number result)
     {
        // ..
        return parsed;
     }
 }

This example seems a little strange, and it is for a reason. I made this code slightly artificial to show the importance of problems appearing in it. First, let’s look at the Parse method. Why should it throw an exception?