Temporal Computation

Computing values that change over time

Time plays a critical role when working with event-based data and event-based models. The ability to calculate point-in-time historical feature values is one of the core features of Kaskada.

Rather than computing a single value, Fenl expressions produce temporal streams describing the result of a given computation as its changes over time.

Solving the Challenge of Event-Based Data

Kaskada is an event-based computation engine. An "event" can be any fact about the world associated with a time, for example, a user signing up for a service, or a customer purchasing a product. Most sources of event-based data change over time as events occur and are added to the system. Computing values from a set of events that changes over time means that the results must change as well.

Traditional data processing systems are designed to answer questions about the current state of a dataset, for instance, "how many purchases has a given user made?". This approach has some drawbacks: the answer to a given question changes based on when it is asked, and the only time at which you can ask questions is "now".

These limitations are reasonable for many use cases, but they make it difficult to build feature examples for training many machine learning models. A common error is accidentally using information that is known "now" to build training examples intended to describe the information available in the past.

The way traditional computations are expressed doesn't help matters. Query languages like SQL and data-processing interfaces like DataFrames were designed to answer questions about tabular (rather than temporal) data. Seemingly simple questions like "how many fraud reports had been filed against each purchase's vendor at the time of purchase?" can require complex windowing and partitioning operations.

How Fenl Deals with Time

Fenl takes a different approach by designing awareness of time into the query language.

Value Streams

Rather than answering a question with a single value, Fenl produces a stream of values describing the answer as it changes over time. For example, the answer to the question "how many purchases has a given user made?" might be the following table:

TimePurchase | count()
2012-02-231
2012-05-102
2018-11-033
2019-10-264
15881588

From this table we can see that if the question was asked in 2015 the answer would be "the user has made two purchases", but if the question was asked now the answer would be "the user has made four purchases".

Final Results

Often it is valuable to know the "final" answer to a question - the last answer that would be produced after all values have been processed. Fenl provides the ability to request final-results for these use cases. The result of query with result behavior set to final-results would be the following:

TimePurchase | count()
2019-10-264
16381638

Final queries make it possible to know the "current" value of a query. Incremental queries and materializations always use final queries.

Temporally-Correct Joins

A core feature of Fenl is the ability to compute temporal joins across datasets. For example the question "how many fraud reports had been filed against each purchase's vendor at the time of purchase?" can be written in a single line.

FraudReport | count() | lookup(Purchase.vendor_id)

© Copyright 2021 Kaskada, Inc. All rights reserved. Privacy Policy

Kaskada products are protected by patents in the United States, and Kaskada is currently seeking protection internationally with pending applications.

Kaskada is a registered trademark of Kaskada Inc.