Blog

A self-adaptive batch request aggregation pattern for improving resource management, response time and costs in microservice and serverless environments

Abstract

Addressing increased demand in cloud environments is typically achieved through auto-scaling the use of further resources allocated. However this comes at the trade-off of increased back-end resource stress as well as runtime costs. In this paper a request batch aggregation and management pattern is proposed and implemented, especially for cases of computational load that needs a heavyweight environment setup (such as performance prediction and AI services). The pattern acts as a preprocessing layer and accumulates requests, withholding their forwarding to the backend based on the conditions of execution (frequency of requests), in order to regulate back-end resource stress without adding extra resources. Different controlling logics are tried out for batch request release, ranging from simpler ones (batch size static regulation, time out interval) to more elaborate ones (ANN model predicting the desired batch size) for adapting the pattern configuration during runtime. Results indicate severe reduction of back-end stress through significantly reducing needed containers (therefore costs as well), enhanced response time as well as avoidance of system breakdown under heavy load. The pattern can be applied in both microservices and serverless environments, especially in Edge cases where resources are constrained.


Authors

  • George Kousiouris

Citation

G. Kousiouris, “A self-adaptive batch request aggregation pattern for improving resource management, response time and costs in microservice and serverless environments”, to appear in the 40th IEEE International Performance Computing and Communications Conference (IPCCC 2021)

You might be interested in …

Newsletter

View our previous Newsletters

Sign up to stay informed on our latest updates!