How data is described and referenced in Fenl.
Fenl operates on typed values. Fenl's type system describes several different kinds of values.
Every expression has an associated type. Since every expression produces a column of values (corresponding to the value at specific points in time), each expression can be thought of as a column of the given type.
Simple values such as the string "hello" or the integer 57 are scalar types. They correspond to a column containing values of the given type (or
Types may be combined to create records. Record fields may be scalar or nested record types. An expression producing a record type is a column that produces a value of the given record type or
null at each point in time.
Scalar types include booleans, numbers, strings, timestamps, durations and calendar intervals.
|Booleans represent true or false|
|Unsigned integer numbers of a particular bit size.|
|Signed integer numbers of a particular bit size.|
|Floating point numbers. When using a decimal a leading numeric character is required.|
|Unicode strings. Strings are written with double-quotes. Double quotes may be escaped within the string.|
|The point in time a given number of seconds, milliseconds, microseconds or nanoseconds after the Unix Epoch (00:00:00 UTC on January 1, 1970).|
|A given number of seconds, milliseconds, microseconds or nanoseconds.|
|A calendrical interval.|
Records allow combining multiple different types into a single value. Records are unnamed - any two records with the same set of fields and value types are considered equal. Fields within a record may have different types. Field names must start with a letter.
|A record is a composite type made up of 0 or (generally) more components. Each component is associated with a|
Fenl implicitly coerces numeric types when different kinds of numbers are combined. For example adding a 64-bit signed integer value to a 32-bit floating point value produces a 64-point floating point value.
Type Promotion Rules
Type coercion will never produce an integer overflow or reduction in numeric precision. Such conversions may be explicitly specified using
The coercion rules can be summarized with the following rules:
- Integers can be widened:
i8 -> i16 -> i32 -> i64.
- Unsigned integers can be widened:
u8 -> u16 -> u32 -> u64.
- Floating point numbers can be widened:
f16 -> f32 -> f64.
- Unsigned integers can be promoted to the next wider integer
u16 -> i32,
u32 -> i64.
- All numbers may be converted to
- Strings may be implicitly converted to timestamps by attempting to parse them as RFC3339 values. The timestamp will be
nullfor strings that don't successfully parse.
Numeric Type Coercion Table
When two numbers are used, Fenl attempts to promote them to a compatible type as the smallest type that both types may be converted to.
The following table shows the result of this promotion for pairs of numeric types.
Coercion to Floating-Point
Note that when
u64is combined with a signed type the result is
f64. This is the only case where an operation between integers produces a floating-point value.
Functions and Signatures
Every function in Fenl has a type signature. For example,
count(input: any, window: window = null) -> u32. This tells us many things about the function:
- It takes two arguments
- The first argument can be of any type (scalar or record).
- The second argument must be a type of
window(the result of a window function such as
- The second argument (
window) is optional, and provides a default value of
- The result is a
Parameters without default values are required. Required arguments may be provided by position or keyword. One required argument may be omitted, in which case it is implicitly
$input. This allows for use of functions with the
| (pipe) syntax. For instance,
TableFoo | count() is treated as
TableFoo | count($input) which is the same as
TableFoo | count(input=$input, window=null).
Parameters with default values in the signature are optional. Arguments for optional parameters must be keyword arguments. For example
count(window = since(...)) but not
When a type constraint (such as
any) appears in a signature, all occurrences of that must be the same type. Type coercion is applied as necessary to make all of the arguments for that constraint compatible.
Additionally, each type constraint imposes restrictions on the types that are valid for arguments with that constraint, as shown in the table below.
|Type Constraint||Valid Types|
|any||Any scalar or record type.|
|key||Any hashable type. This includes |
|number||Any numeric scalar type. This includes |
|signed||Any signed numeric scalar type. This includes |
|float||Any floating point numeric scalar type. This includes |
|timedelta||Any time delta scalar type. This includes |
|ordered||Any ordered scalar type. This includes |
|window||Any result of a window function.|
Updated 11 months ago