Microservice horizontal pattern



The microservice horizontal pattern enables you to run multiple independent models parallelly. You can send one request to the models at once to acquire multiple predictions, or an integrated prediciton. It is possible to respond with aggregated result, or just a specific prediction.
You can decide to run on synchronous or asynchronous depending on your usecase. If you choose to do synchronously, it may be needed to respond the result after aggregated predictions from all the models. If asynchronous, then you can trigger the next action immediately once you get a specific prediction (Asynchronized horizontal).
You may place a proxy in between client and prediction services. You may expect the proxy to control the data retrieval and prediction order away from the client. You may let the proxy or each predictors to do additional data retrieval (Synchronized horizontal with data retrieval). An advantage of proxy to get data is that it will decrease the number of requests to DWH or storage to reduce overhead, while the latter will let you control the data structure depending on each prediction model, that allows you to make complex workflow.


Synchronized horizontal


Synchronized horizontal with data retrieval


Asynchronized horizontal




Needs consideration