Data Engineering *

For several weeks, I logged every moment when understanding our system required asking another person instead of reading code, documentation, dashboards, or configuration. What started as a small experiment turned into a surprisingly useful map of hidden knowledge dependencies. The biggest risk was not missing documentation. It was that some parts of the system effectively existed inside one engineer's head.

I am a contributor to RFC 9562, and I wrote this FAQ because I kept seeing the same concerns about UUIDv7 surfacing across Hacker News, Reddit, and developer blogs. The discussion would always circle back to the same objections, most of which are superficial and fall apart when you look closer. UUIDv7 is too big. The timestamp leaks creation time. It creates hot shards. It’s slower than BIGINT. Sorting breaks if clocks drift. After encountering these objections repeatedly, I decided to collect the most debated questions in one place and answer them based on real implementation experience.

Author: Sergey Lukyanchikov, C-NLTX/Open-Source

Disclaimer: The views expressed in this document reflect the author's subjective perspective on the current and potential capabilities of jBPM.

TL;DR: Zero "quantum supremacy". Zero "agentic orchestration". Zero other hype. Just an approach to achieving an efficient quantum-assisted automation using 100% free open-source components (except for Azure).

In my previous article, I discussed the rationale for adopting jBPM as an AI orchestration platform. This article extends that discussion by examining jBPM’s ability to automate quantum computations and to incorporate their results into business processes and related analytical workflows:

UX audit is a professional review and evaluation of a software product's UX, aimed to identify any types of issues that have a negative impact on the product's performance and provoke user frustration. Its ultimate goal is to provide recommendations on which areas of the product need to be improved to make it more user-oriented and therefore more useful and profitable for the business. Let's discuss how to know when the product needs a UX audit, how to prepare for the process as the product owner, which steps the process consists of and what to do with the results. 

In this article, we will not only install a local (and free) alternative to ChatGPT, but also review several open LLMs, delve into the advanced settings of LM Studio, connect the chatbot to Visual Studio Code, and teach it to assist us with programming. We will also look at how to fine-tune the model's behavior using system prompts.

Hello everyone!

I want to note right away that this article is written exclusively for people who are just starting their journey in learning SQL and window functions. It may not cover complex applications of functions or use complicated definitions—everything is written in the simplest language possible for a basic understanding.

P.S. If the author didn't cover or write about something, it means they considered it non-essential for this article)))

For the examples, we will use a small table that shows student grades in different subjects. In the database, the table looks like this:

Pinterest is a visual discovery platform where people can find ideas like recipes, home and style inspiration, and much more. The platform offers its partners shopping capabilities as well as a significant advertising opportunity with 500+ million monthly active users. Advertisers can purchase ads directly on Pinterest or through partnerships with advertising agencies. Due to our huge scale, advertisers get an opportunity to learn about their Pins and their interaction with Pinterest users from the analytical data. This gives advertisers an opportunity to make decisions which will allow their ads to perform better on our platform.

Hello, Habr! I'd like to share my experience developing such a system.
The defining parameters of a domain-specific system are:

In the big data era, data is one of the most valuable assets for enterprises. The ultimate goal of data analytics is to power swift, agile business decision making. As database technologies advance at a breathtaking pace in recent years, a large number of excellent database systems have emerged. Some of them are impressive in wide-table queries but do not work well in complex queries. Some support flexible multi-table queries but are held back by slow query speed.

Each type of data has a data model that best represents them. However, in real business scenarios, there is no such thing as ultra-fast data analytics under the perfect data model. Big data engineers sometimes have to make compromises on data models. Such compromises may cause long latency in complex queries or damage the real-time query performance because engineers must take the trouble to convert complex data models into flat tables.

New business requirements put forward new challenges for database systems. A good OLAP database system must be able to deliver excellent performance in both wide-table and multi-table scenarios. This system must also reduce the workload of big data engineers and enable customers to query data of any dimension in real time without worrying about data construction.

Apache Druid has been a staple for real-time analytics. However, with evolving and sophisticated analytics demands, it has faced challenges in satisfying modern data performance needs. Enter StarRocks, a high-performance, open-source analytical database, designed to adeptly meet the advanced analytics needs of contemporary enterprises by offering robust capabilities and performance.

In this article, we’ll explore the functionalities, strengths, and challenges of both Apache Druid and StarRocks. Using practical examples and benchmark results, we aim to guide you in identifying which database might best meet your data needs.

The “test everything” principle doesn’t improve data quality — it destroys it. Hundreds of useless alerts create noise that drowns out truly important signals, and the team stops responding to them. Google and Monzo have already moved away from this approach.

Here’s how to shift from blanket testing to targeted checks at nodes with the greatest impact radius — and why one well-placed test at the source is worth more than a hundred checks downstream.

Filename Extension: .6nf

6NF File Format is a new bitemporal, sixth-normal-form (6NF)-inspired data exchange format designed for DWH and for reporting. It replaces complex hierarchical formats like XBRL, XML, JSON, and YAML

This world needs a new theory — a theory that could describe all the theories on the planet. A theory that could easily describe philosophy, mathematics, physics, and psychology. The one that makes all kinds of sciences computable.

This is exactly what we are working on. If we succeed, this theory will become the unified meta-theory of everything.

A year has passed since our last publication, and our task is to share the progress with our English-speaking audience. This is still not a stable version; it’s a draft. Therefore, we welcome any feedback, as well as your participation in the development of the links theory.

As with everything we have done before, the links theory is published and released into the public domain — it belongs to humanity, that means, it is yours. This work has many authors, but the work itself is far more important than any specific authorship. We hope that today it can become useful to more people.

We invite you to become a part of this exciting adventure.

Author: Sergey Lukyanchikov, C-NLTX/Open-Source

Disclaimer: The views expressed in this document reflect the author's subjective perspective on the current and potential capabilities of jBPM.

TL;DR: Zero "agentic AI". Zero "cloud native". Zero other hype. Just an approach to achieving an efficient AI-centric automation using 100% free open-source components.

This text presents jBPM as a platform for orchestrating external AI-centric environments, such as Python, used for designing and running AI solutions. We will provide an overview of jBPM’s most relevant functionalities for AI orchestration and walk you through a practical example that demonstrates its effectiveness as an AI orchestration platform:

In recent years, discussions about the environmental impact of information and communication technologies (ICTs) have largely revolved around hardware — data centers, electronic waste, and energy consumption. However, an equally important factor has been overlooked: the software development methodologies themselves.

When I read the UNCTAD “Digital Economy Report 2024”, I was struck by the complete absence of any mention of how programming methodologies impact sustainability. There was no discussion of whether developers use algorithm-centric or code-centric methodologies when creating software, nor how these choices affect the environment.

This realization led me to introduce the concept of Eco-Methodological Sustainability — a new approach that highlights the role of structured software development methodologies in shaping an environmentally sustainable future for the digital economy.

Apache Kafka is a distributed event-streaming platform designed to handle real-time data feeds. It allows applications to publish, process, and subscribe to streams of data in a highly scalable, fault-tolerant manner.

In today's digital world, where applications process increasing amounts of sensitive data, ensuring reliable user authentication is critical. Authentication is the process of verifying the identity of a user who is trying to access a system. A properly chosen authentication method protects data from unauthorized access, prevents fraud, and increases user confidence.

However, with the development of technology, new authentication methods are emerging, and choosing the optimal solution can be difficult. This article will help developers and business owners understand the variety of authentication approaches and make informed choices.

Prerequisites:
1. Install Docker
2. Install VSCode

STEP BY STEP

1. Open VSCode that you previously installed and click on "Extensions" tab right on the menu bar, then type 'docker' to find proper extension and click "install":

Wearable Digital Health Technologies for Monitoring in Cardiovascular Medicine

This review article presents a three-part true-life clinical vignette that illustrates how digital health technology can aid providers caring for patients with cardiovascular disease. Specific information that would identify real patients has been removed or altered. Each vignette is followed by a discussion of how these methods were used in the care of the patient.

Below, we will discuss user-defined aggregation functions (UDAF) using org.apache.spark.sql.expressions.Aggregator, which can be used for aggregating groups of elements in a DataSet into a single value in any user-defined way.

Let’s start by examining an example from the official documentation that implements a simple aggregation