Managing caching strategies for stream reasoning with reinforcement
  learning
  Efficient decision-making over continuously changing data is essential for
many application domains such as cyber-physical systems, industry
digitalization, etc. Modern stream reasoning frameworks allow one to model and
solve various real-world problems using incremental and continuous evaluation
of programs as new data arrives in the stream. Applied techniques use, e.g.,
Datalog-like materialization or truth maintenance algorithms to avoid costly
re-computations, thus ensuring low latency and high throughput of a stream
reasoner. However, the expressiveness of existing approaches is quite limited
and, e.g., they cannot be used to encode problems with constraints, which often
appear in practice. In this paper, we suggest a novel approach that uses the
Conflict-Driven Constraint Learning (CDCL) to efficiently update legacy
solutions by using intelligent management of learned constraints. In
particular, we study the applicability of reinforcement learning to
continuously assess the utility of learned constraints computed in previous
invocations of the solving algorithm for the current one. Evaluations conducted
on real-world reconfiguration problems show that providing a CDCL algorithm
with relevant learned constraints from previous iterations results in
significant performance improvements of the algorithm in stream reasoning
scenarios.
  Under consideration for acceptance in TPLP.
