What is an "Expression evaluator"?

Expression evaluator allows you to evaluate specified expressions, for example:

12*3 is 36
[1,2,3].reverse() is an array of [3,2,1]
'Kate'.reverse() is "etaK"

Expressions may depend on input variables:

10*x + 4 depends on the value of x.
'My name is {userName}. Age is {2022-birthYear}' depends on the userName and birthYear values.

A script can contain several such expressions:

x = Vx*t
y = Vy*t

distance = sqrt(x**2 + y**2)
average = (x+y)/2

In general, you can use Nfun wherever you have previously stored, passed, or configured constants. In the example below, we formally set the rules for calculating bonuses

// ...`settings.json`...
{
    "offset": "25",
    "timeOffset": "3* 60 * 60 * 24 #sec",
    "condition": "if (age>18) isEmpty(orders) else isEmpty(parent.orders)",
    "bonus": "(min(max(order.price, 20.0), 100) + prevBonus)/ordersCount"
}

Here are some usage examples:

Backend: Incoming request filters;
Embedded: configuring signal processing;
Loyalty system: bonus program settings;
Robotics: kinematic model. Trajectory Descriptions;
Low-code solutions.

What can NFun do

Nfun continues the idea of the Ncalc library, but for a rich type system

Primitive types - byte, u16, u32, u64, i16, i32, i64, real, bool, ip, char, text, any
Arrays, structures, Lambda expressions and Linq.
Arithmetic, binary, discrete operators, array operators
Conditional if expression.
String interpolation.
Named expressions and user-defined functions.
Strict type system with type inference.
Built-in functions.
semantics customization

Playground

Install the nuget package NFun

PM> Install-Package NFun

Let's start with the classics!

var a = Funny.Calc("'Hello world'");
Console.WriteLine(a);

Let 's calculate the constants

bool   b = Funny.Calc<bool>("false and (2 > 1)"); // false of bool  
double d = Funny.Calc<double>(" 2 * 10 + 1 "); // 21  of double  
int    i = Funny.Calc<int>(" 2 * 10 + 1 "); // 21 of int

Let's calculate the output data

class User { public string Age {get;set;} public string Name {get;set;} }

var inputUser = new User{ Age = 42; Name = "Ivan"; }

string userAlias = 
    Funny.Calc<User,string> ( 
        "if(age < 18) name else 'Mr. {name}' ", 
        inputUser);

And now let's switch to hardcore mode. This mode provides access to all variables, and to low-level execution control.

var runtime = Funny.Hardcore.Build(
    "y = a-b; " +
    "out = 2*y/(a+b)"
);
// Set inputs
runtime["a"].Value = 30;
runtime["b"].Value = 20;
// Run script
runtime.Run();
// Get outputs
Assert.AreEqual(0.4, runtime["out"].Value);

We can continue...

Calculating multiple values based on input variable

// Assume we have some С# model
/*
    class SomeModel {
        public SomeModel(int age, Car[] cars) {
            Age = age;
            Cars = cars;
        }
        public int Age { get; }    //Used as input
        public Car[] Cars { get; } //Used as input
        public bool Adult { get; set; }   //Used as output
        public double Price { get; set; } //Used as output
    }
*/

var context =  new SomeModel(
  age:42, 
  cars: new []{ new Car{ Price = 6000 }, new Car{ Price = 6200 }}
);

// then we can set the 'Adult' and 'Price' properties based on the value of the 'Age' and 'Cars' properties
Funny.CalcContext(
    @"  
        adult = age>18
        price = cars.sum(rule it.price)
    ",
    context);
Assert.AreEqual(true, context.Adult);
Assert.AreEqual(12200, context.Price);
// So input values and output values are properties of the same object

Customization

Nfun provides customization of syntax and semantics for your needs

Prohibition or permission of if-expressions
Decimal or Double arithmetic
Integer overflow behavior
Prohibition or permission of custom functions
Default type for integer constants

var uintResult = Funny
        .WithDialect(integerOverflow: IntegerOverflow.Unchecked)
        .Calc<uint>("0xFFFF_FFFF + 1");
    Assert.AreEqual((uint)0, uintResult);


//now you cannot launch script with such an expression
var builder = Funny.WithDialect(IfExpressionSetup.Deny);
Assert.Throws<FunnyParseException>(
  () => builder.Calc("if(2<1) true else false"));

Add functions and constants

//assume we have custom function (method or Func<...>)
Func<int, int, int> myFunctionMin = (i1, i2) => Math.Min(i1, i2);

object a = Funny
            .WithConstant("foo", 42)
            .WithFunction("myMin", myFunctionMin)
            // now you can use 'myMin' function and 'foo' constant in script!
            .Calc("myMin(foo,123) == foo");
Assert.AreEqual(true, a);

Syntax

Nfun supports single-line expressions:

12 * x**3 - 3

Multiline named expressions:

nameStr = 'My name is: "{name}"'
ageStr = 'My age is {age}'
result = '{nameStr}. {ageStr}'

And custom functions:

maxOf3(a,b,c) = max(max(a,b),c)

y = maxOf3(1,2,3) # 3

Depending on the task, you can enable and disable these features.

More about the syntax

Operators

# Arithmetic operators: + - * / % // ** 
y1 = 2*(x//2 + 1) / (x % 3 -1)**0.5

# Bitwise:     ~ | & ^ << >>
y2 = (x | y & 0xF0FF << 2) ^ 0x1234

# Discreet:    and or not > >= < <= == !=
y3 = x and false or not (y>0)

if-expression

simple  = if (x>0) x else if (x==0) 0 else -1
complex = if (age>18)
            if (weight>100) 1
            if (weight>50)  2
            else 3
        if (age>16) 0
        else       -1

User functions and generic arithmetic

sum3(a,b,c) = a+b+c #define generic user function sum3

r:real = sum3(1,2,3) 
i:int  = sum3(1,2,3)

Arrays

# Initialization
a:int[] = [1,2,3,4]      # [1,2,3,4]  type: int[]
b = ['a','b','foo']# ['a','b','foo'] type: text[]
c = [1..4] 	   # [1,2,3,4]  type: int[]
d = [1..7 step 2]      # [1,3,5,7]  type: int[]

# Operator in
a = 1 in [1,2,3,4]		# true

# Get values
c = (x[5]+ x[4])/3

# Slices
y = [0..10][1:3] #[1,2,3]
y = [0..10][7:]  #[7,8,9,10]
y = [0..10][:2]  #[0,1,2]
y = [0..10][1:5 step 2] #[1,3,5]

# Functions
# concat, intersect, except, unite, unique, find, max, min, avg, median, sum, count, any, sort, reverse, chunk, fold, repeat

Structures

# initialization
user = {
    age = 12, 
    name = 'Kate'
    cars = [ # array of structures
        { name = 'Creta',   id = 112, power = 140, price = 5000},
        { name = 'Camaro', id = 113, power = 353, price = 10000} 
    ]
}
userName = user.name # field access

Strings

a =  ['one', 'two', 'three'].join(', ') # "one, two, three" of String

# Interpolation:
x = 42
out = '21*2= {x}, arr = {[1,2,x]}' 
#"21*2= 42, arr = [1,2,42]" of String

Anonymous-functions

[1,2,3,4]
    .filter(rule it>2)
    .map(rule it**3)
    .max() # 64

Semantics

Nfun is strictly typed - this was the main challenge, and a key feature for integration into C#, as well as error protection. However, the syntax of strongly typed languages is always more complicated.

To solve this problem, I relied on the postulate:

Anything that looks like a proper script (within the framework of syntax/semantics) should run

Or, more formalized:

If the code looks like a weakly typed script, but it runs without errors, then it can be unambiguously output types for it.
Or to show that such code cannot be executed without errors.

This required the development of a complex type inference system, from which the entire semantics of the language is based. The result of this is the widespread use of Generic-types.

In the example below, functions, calculations, and even constants are generalized types (from the list int32, uint32, int64, uint64, real). However, the user should not think about it:

var expr = @"
  # generic function
  inc(a) = a + 1  

  # generic calculation
  out = 42.inc().inc()
";
double d = Funny.Calc<double>(expr); // 44  of double  
int    i = Funny.Calc<int>(expr); // 44 of int

Thus, it was possible to collect all the advantages of strict typing:

If the expression types do not converge, you get an error at the interpretation stage.
If the types do not converge with the expected C# types, you get an error at the interpretation stage.
Perfomance

At the same time, the syntax remained as simple as possible for an usual user!

Technical Details

Since Nfun is "almost a programming language" under the hood, its architecture is fairly standard. I will not describe it in detail here, there are already many cool articles about it.

Code interpretation can be divided into several stages:

Tokenization (lexer).

A self-written lexer parses the input string into tokens, taking into account the possibility of string interpolation.

Parsing.

Parsing tokens into the AST tree. A self-written parser is used. The syntax specification relies on the language specification (and a lot of tests). The formal grammar is not described.

Type inference.

A custom, graph-based type inference algorithm, with support for implicit conversions between types.

Type inference nuances:

Integer constants are "generalized constants"

one() = 1 # returns T, where byte -> T -> real 

y:int  = one() # 1 of int
z:real = one() # 1.0 of real

The node type may depend on both the previous and subsequent code

y = 1 # y:real, так как используется в знаменателе на следующей строчке
x:real = 1/y

Restrictions on generic variables can be set both from above (inheritance, the type to which this can be reduced) and from below (descendant, the type that can be reduced to this)

sum(a,b) = a+b

x:int  = 1 + 2  # x = 3 of int
y:real = 1 + 2 # y = 3.0 of real
z:byte = 1 + 2 # error! operator '+' is defined for (uint16|int16)-> T -> real
               # so it cannot return 'byte'

Assembly of expressions (construction of runtime)

A self-written tree of calculated expressions is assembled from the results of solving types and the Ast tree.

Conversion of CLR values to Nfun.
Execution of expressions.
Conversion of results to CLR values.

I deliberately did not use the Csharp-expression-tree, since one of the most important criteria was the "one-time-shot" speed. That is, the launch, from the moment the script line is received until the execution results are received.

Project status

The project is ready for use in production.

It has been used for a year as part of a scada system, covered with 6000+ tests, a specification has been written for it, and .. there are even a few stars on the github (yes, it's a call to action!). Almost a success!

Conclusion

When I started Nfun, I dreamed of creating a simple, reliable and intuitive open source tool. I wanted to implement syntactic ideas and experiment with type inference systems, try something new for myself...

As a result, Nfun cost a huge amount of effort, time and investment. This is the first time I have faced such a task. Now I want people to use this tool! And they wrote tickets and requests for github. Well, or comments under this post ;)

NFun — expression evaluator for .Net