Xmibw3oxSDmPYljzijxko+ds8evW6l6tbcKcgi8Qp ShQKNbuWr0tYFqM0TFCt256bmiCnynAmcFzuZBpTyoa0j21LJY1RB/n02DE5tkqPRImyJQ2Zqr8nchprPYpD2xlTM9Dz3kT8z2tnJroMci7TzKBks0VRJohJyORz0uMKmREjSyhT3N5K2IAqyozNp2xD8OZfXiT+ae2q5t2dV+vXR Xmibw3oxSDmPYljzijxko+ds8evW6l6tbcKcgi8Qp ShQKNbuWr0tYFqM0TFCt256bmiCnynAmcFzuZBpTyoa0j21LJY1RB/n02DE5tkqPRImyJQ2Zqr8nchprPYpD2xlTM9Dz3kT8z2tnJroMci7TzKBks0VRJohJyORz0uMKmREjSyhT3N5K2IAqyozNp2xD8OZfXiT+ae2q5t2dV+vXR Xmibw3oxSDmPYljzijxko+ds8evW6l6tbcKcgi8Qp ShQKNbuWr0tYFqM0TFCt256bmiCnynAmcFzuZBpTyoa0j21LJY1RB/n02DE5tkqPRImyJQ2Zqr8nchprPYpD2xlTM9Dz3kT8z2tnJroMci7TzKBks0VRJohJyORz0uMKmREjSyhT3N5K2IAqyozNp2xD8OZfXiT+ae2q5t2dV+vXR

2、每次数据D致都要 MERGE 存量数据 。T+ 方GT新3R效性差。 3、不M持CR1ps+rt。 缺点 SCaDk + )=AFa IL()(数据 MER,E .NTO GE=DE US.N, chan>=E ON GE=DE.GE=D.< = chan>=E.GE=D.< WHEN MAT(HE) AN) +LA,=H)H THEN )ELETE WHEN MAT(HE) AN) +LA,<>H)H I (1,2 I (1,2 D (1,2 I (1,2 D (1,2 I (1,2 IN-E,T(1,2 DE(ETE(1,2 IN-E,T(1,2 问题：)1rg1-On-,1adL终读取RM为空集，实际I该返E I(1,2 。 IN-E,T DE(ETE data 2il11 data 2il11 1quality d1l1t1 2il11 data 2il11 1quality (1,3 , (1,4 , (1,3 , (1,4 , (1,( 53t3 file2 , (1,2 DE-E2E (1,( 53t3 file1 , (1,3 , (1,4 , (1,( 53t3 file2 , (1,2 DE-E2E (1,4 53t3 file1 , (1,3 , (1,4 (file2, 0 , (1,( 53t3

much input do we need to re-play? How expensive is it to re-construct the state? How fast can we de-duplicate output? Vasiliki Kalavri | Boston University 2020 Gap Recovery • Restart the operator net/fig/5-2 • Receivers store a catalog of all identifiers they have seen and processed. • The de-duplication catalog is stored in a scalable key/value store. Vasiliki Kalavri | Boston University Bloom filters for performance • Maintaining a catalog of all IDs ever seen and checking it for de-duplication is expensive • In a healthy pipeline though, most records will not be duplicates •

Stefani, Lorenzo De, et al. Triest: Counting local and global triangles in fully dynamic streams with fixed memory size. TKDD 2017. https://www.kdd.org/ kdd2016/papers/files/rfp0465-de-stefaniA.pdf Further

unsubscribe advertise(): information reg. future events Publish/Subscribe Systems 17 Pub/Sub levels of de-coupling • Space: interacting parties do not need to know each other • Publishers do not know who

University 2020 RocksDBStateBackend • Stores all state into embedded RocksDB instances • Accesses require de/serialization • Checkpoints state to a remote file system and supports incremental checkpoints •

in a sep- arate Flink cluster. Flink is responsible for talking with Kubernetes and allocating and de-allocating TaskManagers depending on the required resources. ./bin/flink run-application \ --target

in a sep- arate Flink cluster. Flink is responsible for talking with Kubernetes and allocating and de-allocating TaskManagers depending on the required resources. ./bin/flink run-application \ --target