Data Model
How data is described and referenced in Fenl.
Types
Fenl operates on typed values. Fenl's type system describes several different kinds of values.
Every expression has an associated type. Since every expression produces a column of values (corresponding to the value at specific points in time), each expression can be thought of as a column of the given type.
Simple values such as the string "hello" or the integer 57 are scalar types. They correspond to a column containing values of the given type (or null
).
Types may be combined to create records. Record fields may be scalar or nested record types. An expression producing a record type is a column that produces a value of the given record type or null
at each point in time.
Scalars
Scalar types include booleans, numbers, strings, timestamps, durations and calendar intervals.
Type | Examples | Description |
---|---|---|
|
| Booleans represent true or false |
|
| Unsigned integer numbers of a particular bit size. |
|
| Signed integer numbers of a particular bit size. |
|
| Floating point numbers. When using a decimal a leading numeric character is required. |
|
| Unicode strings. Strings are written with double-quotes. Double quotes may be escaped within the string. |
|
| The point in time a given number of seconds, milliseconds, microseconds or nanoseconds after the Unix Epoch (00:00:00 UTC on January 1, 1970). |
|
| A given number of seconds, milliseconds, microseconds or nanoseconds. |
|
| A calendrical interval. |
Records
Records allow combining multiple different types into a single value. Records are unnamed - any two records with the same set of fields and value types are considered equal. Fields within a record may have different types. Field names must start with a letter.
Type | Examples | Description |
---|---|---|
|
| A record is a composite type made up of 0 or (generally) more components. Each component is associated with a |
Type Coercion
Fenl implicitly coerces numeric types when different kinds of numbers are combined. For example adding a 64-bit signed integer value to a 32-bit floating point value produces a 64-point floating point value.
Type Promotion Rules
Type coercion will never produce an integer overflow or reduction in numeric precision. Such conversions may be explicitly specified using as
.
The coercion rules can be summarized with the following rules:
- Integers can be widened:
i8 -> i16 -> i32 -> i64
. - Unsigned integers can be widened:
u8 -> u16 -> u32 -> u64
. - Floating point numbers can be widened:
f16 -> f32 -> f64
. - Unsigned integers can be promoted to the next wider integer
u8
->i16
,u16 -> i32
,u32 -> i64
. - All numbers may be converted to
f64
. - Strings may be implicitly converted to timestamps by attempting to parse them as RFC3339 values. The timestamp will be
null
for strings that don't successfully parse.
Numeric Type Coercion Table
When two numbers are used, Fenl attempts to promote them to a compatible type as the smallest type that both types may be converted to.
The following table shows the result of this promotion for pairs of numeric types.
i8 | i16 | i32 | i64 | u8 | u16 | u32 | u64 | f16 | f32 | f64 | |
---|---|---|---|---|---|---|---|---|---|---|---|
i8 | i8 | i16 | i32 | i64 | i16 | i32 | i64 | f64 | f16 | f32 | f64 |
i16 | i16 | i16 | i32 | i64 | i16 | i32 | i64 | f64 | f16 | f32 | f64 |
i32 | i32 | i32 | i32 | i64 | i16 | i32 | i64 | f64 | f16 | f32 | f64 |
i64 | i64 | i64 | i64 | i64 | i64 | i64 | i64 | f64 | f16 | f32 | f64 |
u8 | i16 | i16 | i16 | i64 | u8 | u16 | u32 | u64 | f16 | f32 | f64 |
u16 | i32 | i32 | i32 | i64 | u16 | u16 | u32 | u64 | f16 | f32 | f64 |
u32 | i64 | i64 | i64 | i64 | u32 | u32 | u32 | u64 | f16 | f32 | f64 |
u64 | f64 | f64 | f64 | f64 | u64 | u64 | u64 | u64 | f16 | f32 | f64 |
f16 | f16 | f16 | f16 | f16 | f16 | f16 | f16 | f16 | f16 | f32 | f64 |
f32 | f32 | f32 | f32 | f32 | f32 | f32 | f32 | f32 | f32 | f32 | f64 |
f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 |
️ Coercion to Floating-Point
Note that when
u64
is combined with a signed type the result isf64
. This is the only case where an operation between integers produces a floating-point value.
Functions and Signatures
Every function in Fenl has a type signature. For example, count(input: any, window: window = null) -> u32
. This tells us many things about the function:
- It takes two arguments
input
andwindow
. - The first argument can be of any type (scalar or record).
- The second argument must be a type of
window
(the result of a window function such assince
orsliding
). - The second argument (
window
) is optional, and provides a default value ofnull
. - The result is a
u32
.
Parameters without default values are required. Required arguments may be provided by position or keyword. One required argument may be omitted, in which case it is implicitly $input
. This allows for use of functions with the |
(pipe) syntax. For instance, TableFoo | count()
is treated as TableFoo | count($input)
which is the same as TableFoo | count(input=$input, window=null)
.
Optional Parameters
Parameters with default values in the signature are optional. Arguments for optional parameters must be keyword arguments. For example count(window = since(...))
but not count(since(...))
.
Type Constraints
When a type constraint (such as any
) appears in a signature, all occurrences of that must be the same type. Type coercion is applied as necessary to make all of the arguments for that constraint compatible.
Additionally, each type constraint imposes restrictions on the types that are valid for arguments with that constraint, as shown in the table below.
Type Constraint | Valid Types |
---|---|
any | Any scalar or record type. |
key | Any hashable type. This includes bool , i8 , i16 , i32 , i64 , u8 , u16 , u32 , u64 and string |
number | Any numeric scalar type. This includes i8 , i16 , i32 , i64 , u8 , u16 , u32 , u64 , f16 , f32 and f64 . |
signed | Any signed numeric scalar type. This includes i8 , i16 , i32 , i64 , f16 , f32 and f64 . |
float | Any floating point numeric scalar type. This includes f16 , f32 and f64 . |
timedelta | Any time delta scalar type. This includes duration_s , duration_ms , duration_us , duration_ns , interval_days and interval_months . |
ordered | Any ordered scalar type. This includes i8 , i16 , i32 , i64 , u8 , u16 , u32 , u64 , f16 , f32 , f64 , timestamp_s , timestamp_ms , timestamp_us , and timestamp_ns . |
window | Any result of a window function. |
Updated 2 months ago