All streams

Asynchronous I/O, ML-based query plan optimization, and built-in connection pooling are among the key features of the new Postgres Pro Enterprise 18. This release brings together the capabilities of the vanilla PostgreSQL 18 core with Enterprise-grade tools for working with large-scale data. Today we will walk through the technical details, new index scanning strategies, and mechanisms for scaling write workloads.

In modern neural networks, including Transformer-based LLMs, unbounded activation functions—ReLU and GELU—have become the standard. Their main advantage is good gradient flow and the rapid training of deep models.

However, in practice, a problem is observed: when dominant patterns or high-frequency noise appear in the input context (long dialogues, noisy data, repetitive or dominant tokens), models become unstable and prone to generation degradation and hallucinations.

In this article, I attempted to find out if the choice of activation function could be fundamentally linked to LLM hallucinations.