For smart decision making, user agents need real-time and historic access to open data from sensors installed in the public domain. In contrast to a closed environment, for Open Data and federated query processing algorithms, the data publisher cannot anticipate in advance on specific questions, nor can it deal with a bad cost-efficiency of the server interface when data consumers increase. When publishing observations from sensors, different fragmentation strategies can be thought of depending on how the historic data needs to be queried. Furthermore, both publish/subscribe and polling strategies exist to publish real-time updates. Each of these strategies come with their own trade-offs regarding cost-efficiency of the server-interface, user-perceived performance and CPU use. A polling strategy where multiple observations are published in a paged collection was tested in a proof of concept for parking space availabilities. In order to understand the different resource trade-offs presented by publish/subscribe and polling publication strategies, we devised an experiment on two machines, for a scalability test. The preliminary results were inconclusive and suggest more large scale tests are needed in order to see a trend. While the large-scale tests will be performed in future work, the proof of concept helped to identify the technical Open Data principles for the 13 biggest cities in Flanders.
Published in 2018 in Proceedings of the Web Stream Processing Workshop.
J. Rojas Meléndez, B. Van de Vyvere, A. Gevaert, R. Taelman, P. Colpaert, and R. Verborgh, “A preliminary Open Data publishing strategy for real-time data in Flanders,” in Proceedings of the Web Stream Processing Workshop, 2018, pp. 1847–1853.
Julián Rojas Meléndez, Brecht Van de Vyvere, Arne Gevaert, Ruben Taelman, Pieter Colpaert, and Ruben Verborgh. 2018. A preliminary Open Data publishing strategy for real-time data in Flanders. In Proceedings of the Web Stream Processing Workshop. 1847–1853.
Rojas Meléndez, J., Van de Vyvere, B., Gevaert, A., Taelman, R., Colpaert, P., Verborgh, R.: A preliminary Open Data publishing strategy for real-time data in Flanders. In: Proceedings of the Web Stream Processing Workshop. pp. 1847–1853 (2018).
Rojas Meléndez, J., Van de Vyvere, B., Gevaert, A., Taelman, R., Colpaert, P., & Verborgh, R. (2018). A preliminary Open Data publishing strategy for real-time data in Flanders. In Proceedings of the Web Stream Processing Workshop (pp. 1847–1853).
Rojas Meléndez, Julián et al. “A Preliminary Open Data Publishing Strategy for Real-Time Data in Flanders.” Proceedings of the Web Stream Processing Workshop. 2018. 1847–1853. Print.