The German Research Center for Artificial Intelligence (DFKI) carried out the 3-week tests (from September 4 to 23, 2023) of the PHYSICS platform to assess the goals targeted by the Smart Manufacturing Use Cases:

1. Deployment of substitute service in the Cloud
2. High Confidence Quality Control

(see Blogpost: Industrial Use Cases of FaaS: The basics you need to know for more details)

The evaluation phase was held from 8:00 a.m. to 8:00 p.m., Monday through Saturday. The test consisted of calling the High Confidence Quality Control PHYSICS Cloud deployed function, resulting in approximately 20000 requests per week. This gave the total number of 60177 requests with 59425 successful responses (98.75%), even though PHYSICS is still in beta testing.

Only one major problem with the PHYSICS platform occurred on Monday afternoon during the first week of testing, which, once noticed, was fixed relatively quickly on Tuesday. This can be seen in the weekly results where the successful response rate was 96.47% in the first week, 99.82% in the second week and 99.96% in the third week.

Considering the first Smart Manufacturing Use Case PHYSICS Cloud Deployment is only meant to be used as a backup solution in case when the DFKI’s PHYSICS Edge Deployment goes out of service. Therefore, the availability of the PHYSICS platform (percentage of successful responses) at this stage of development was assessed as satisfactory.

In addition, for every single request the action time[1] and the total time[2] were recorded. Both Smart Manufacturing Use Cases required an average of less than 5 seconds for a Quality Check. The performance of PHYSICS fulfilled this condition which can be seen in Figure 1.

[1] only function execution time

[2] total request round-trip time

It is important to highlight that the total time for the successful request is consistent as shown in the histogram presented in Figure 2. The only outliers are the required cold starts of the functions each day after not being used overnight (not shown in the histogram, since they last longer than 10 seconds).

Because of the pay-as-you-go model of Function as a Service in general, our secondary goal of reducing costs was also achieved. About 20000 requests per week, each lasting less than 2 seconds would cost less than 1€ per month in operational costs[1]. Of course, if we need more requests or even more complex quality checks, operating time will increase, and therefore cost will scale linearly with it. Due to the simplified development and deployment, replacing our functions in the future will be straightforward, further reducing (development) costs.

To sum up, the tests conducted for Smart Manufacturing Use Cases proved the potential of the PHYSICS. Considering that the platform is still at the testing stage, its availability, performance, and chance to reduce development costs were evaluated as satisfactory.

[1] Estimated cost based on comparable FaaS platforms on the market.

The post Assessment of goals targeted by Smart Manufacturing Use Cases appeared first on PHYSICS.