Extreme-scale Cloud/Fog Data Processing refers to the capability of processing exceptionally large and complex datasets in cloud and fog computing environments, with a particular emphasis on scalability and efficiency. To achieve scalable processing the EMERALDS services are implemented using state-of-the-art big data frameworks, such as Apache Spark and Apache Kafka. In particular, the EMERALDS services under this toolset are tailored for urban mobility data. Noteworthy results include a novel scalable algorithm for parallel spatial joins that exploits adaptive replication, which can be used to associate different kinds of spatial data based on distance at scale. Also, algorithms for parallel processing of complex spatial data that include text are offered. This includes both exact keyword matching as well as semantic search based on vectors (embeddings) that represent the textual information. Moreover, a weather enrichment service is provided that can integrate spatio-temporal positions with weather data that is available by third-part services. Last, but not least, a method for hot-spot analysis over road networks is proposed that takes into account GPS traces of moving vehicles so as to identify statistically significant hot spots in the form of road segments in an urban environment.
