We have seen how Apache Spark can be used for processing batch (Spark Core) as well as real-time data (Spark Streaming). Sometimes the data we need to deal with is connected in nature. For example ...

Thus it is a sequence of discrete-time data. Examples of time series are heights of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.

The “Map” phase processes and filters data, while the “Reduce” phase aggregates the results, making it suitable for a wide range of data processing tasks. Watch this video on YouTube.

Eventual built a Python-native open source data processing engine, known as Daft, that is designed to work quickly across different modalities from text to audio and video, and more.