Data Model

How data is described and referenced in Fenl.

Types

Fenl operates on typed values. Fenl's type system describes several different kinds of values.

Every expression has an associated type. Since every expression produces a column of values (corresponding to the value at specific points in time), each expression can be thought of as a column of the given type.

Simple values such as the string "hello" or the integer 57 are scalar types. They correspond to a column containing values of the given type (or null).

Types may be combined to create records. Record fields may be scalar or nested record types. An expression producing a record type is a column that produces a value of the given record type or null at each point in time.

Scalars

Scalar types include booleans, numbers, strings, timestamps, durations and calendar intervals.

TypeExamplesDescription
booltrue, falseBooleans represent true or false
u8, u32, u640, 1, 10000Unsigned integer numbers of a particular bit size.
i8, i32, i640, 1, -100, 10000, 0.0, -1.0Signed integer numbers of a particular bit size.
f32, f640, 1, -100, 10000, 0.0, -1.0, -100837.631Floating point numbers. When using a decimal a leading numeric character is required.
string"hello", "hello \"john\""Unicode strings. Strings are written with double-quotes. Double quotes may be escaped within the string.
timestamp_s, timestamp_ms, timestamp_us, timestamp_ns1639595174 as timestamp_sThe point in time a given number of seconds, milliseconds, microseconds or nanoseconds after the Unix Epoch (00:00:00 UTC on January 1, 1970).
duration_s, duration_ms, duration_us, duration_ns0 as duration_s, 1 as duration_ms, -100 as duration_us, 10000 as duration_nsA given number of seconds, milliseconds, microseconds or nanoseconds.
interval_days, interval_months0 as interval_days, 1 as interval_days, -100 as interval_months, 10000 as interavl_monthsA calendrical interval.

Records

Records allow combining multiple different types into a single value. Records are unnamed - any two records with the same set of fields and value types are considered equal. Fields within a record may have different types. Field names must start with a letter.

TypeExamplesDescription
{name: string, age: number}{name: "john", age: 32}A record is a composite type made up of 0 or (generally) more components. Each component is associated with a
field name.

Type Coercion

Fenl implicitly coerces numeric types when different kinds of numbers are combined. For example adding a 64-bit signed integer value to a 32-bit floating point value produces a 64-point floating point value.

Type Promotion Rules

Type coercion will never produce an integer overflow or reduction in numeric precision. Such conversions may be explicitly specified using as.

The coercion rules can be summarized with the following rules:

  1. Integers can be widened: i8 -> i16 -> i32 -> i64.
  2. Unsigned integers can be widened: u8 -> u16 -> u32 -> u64.
  3. Floating point numbers can be widened: f16 -> f32 -> f64.
  4. Unsigned integers can be promoted to the next wider integer u8 -> i16, u16 -> i32, u32 -> i64.
  5. All numbers may be converted to f64.
  6. Strings may be implicitly converted to timestamps by attempting to parse them as RFC3339 values. The timestamp will be null for strings that don't successfully parse.

Numeric Type Coercion Table

When two numbers are used, Fenl attempts to promote them to a compatible type as the smallest type that both types may be converted to.
The following table shows the result of this promotion for pairs of numeric types.

i8i16i32i64u8u16u32u64f16f32f64
i8i8i16i32i64i16i32i64f64f16f32f64
i16i16i16i32i64i16i32i64f64f16f32f64
i32i32i32i32i64i16i32i64f64f16f32f64
i64i64i64i64i64i64i64i64f64f16f32f64
u8i16i16i16i64u8u16u32u64f16f32f64
u16i32i32i32i64u16u16u32u64f16f32f64
u32i64i64i64i64u32u32u32u64f16f32f64
u64f64f64f64f64u64u64u64u64f16f32f64
f16f16f16f16f16f16f16f16f16f16f32f64
f32f32f32f32f32f32f32f32f32f32f32f64
f64f64f64f64f64f64f64f64f64f64f64f64

ℹ️

Coercion to Floating-Point

Note that when u64 is combined with a signed type the result is f64. This is the only case where an operation between integers produces a floating-point value.

Functions and Signatures

Every function in Fenl has a type signature. For example, count(input: any, window: window = null) -> u32. This tells us many things about the function:

  1. It takes two arguments input and window.
  2. The first argument can be of any type (scalar or record).
  3. The second argument must be a type of window (the result of a window function such as since or sliding).
  4. The second argument (window) is optional, and provides a default value of null.
  5. The result is a u32.

Parameters without default values are required. Required arguments may be provided by position or keyword. One required argument may be omitted, in which case it is implicitly $input. This allows for use of functions with the | (pipe) syntax. For instance, TableFoo | count() is treated as TableFoo | count($input) which is the same as TableFoo | count(input=$input, window=null).

Optional Parameters

Parameters with default values in the signature are optional. Arguments for optional parameters must be keyword arguments. For example count(window = since(...)) but not count(since(...)).

Type Constraints

When a type constraint (such as any) appears in a signature, all occurrences of that must be the same type. Type coercion is applied as necessary to make all of the arguments for that constraint compatible.

Additionally, each type constraint imposes restrictions on the types that are valid for arguments with that constraint, as shown in the table below.

Type ConstraintValid Types
anyAny scalar or record type.
keyAny hashable type. This includes bool, i8, i16, i32, i64, u8, u16, u32, u64 and string
numberAny numeric scalar type. This includes i8, i16, i32, i64, u8, u16, u32, u64, f16, f32 and f64.
signedAny signed numeric scalar type. This includes i8, i16, i32, i64, f16, f32 and f64.
floatAny floating point numeric scalar type. This includes f16, f32 and f64.
timedeltaAny time delta scalar type. This includes duration_s, duration_ms, duration_us, duration_ns, interval_days and interval_months.
orderedAny ordered scalar type. This includes i8, i16, i32, i64, u8, u16, u32, u64, f16, f32, f64, timestamp_s, timestamp_ms, timestamp_us, and timestamp_ns.
windowAny result of a window function.

© Copyright 2021 Kaskada, Inc. All rights reserved. Privacy Policy

Kaskada products are protected by patents in the United States, and Kaskada is currently seeking protection internationally with pending applications.

Kaskada is a registered trademark of Kaskada Inc.