Performance of STS RSFN CLUSTER 7.0.3

Scalability refers to the ability to do more work in proportion to the increase in capacity of the system doing the work.

When we take a certain system that we know does work on a machine with one processor, we expect that same system to do twice as much work on a machine with two processors.

In practice, due to the existence of various types of bottleneck in the system, from the hardware to the operating system to the application, real scalability will be lower.

The aim of this work is to evaluate the scalability of an STS cluster in terms of the concurrency variable, i.e. the amount of work being done concurrently.

By concurrency level we mean the amount of work competing for the system, i.e. running "at the same time"; this could mean an application with N threads running synchronous operations or it could mean an application running asynchronous operations with N pending operations at the same time.

Taking a fixed architecture, we carried out measurements by varying the amount of concurrency, in order to check the impact on the latency (time of an operation) and throughput (rate of operations per unit of time) metrics.

In this measurement, we used an architecture with two STS Servers running on AWS EC2 machines and a Java STS Client running on a third AWS EC2 machine .

We obtained each metric by performing the operation on a file of 1 KiB in size, configuring the measurement for the concurrency level in the range 1, 2, 4, 8, 16, 32, 64, 128, 256, 384 and 512.

RESULTS:

We see that the flow curve rises consistently with increasing competition until competition level 256, when it starts to fall, and the latency curves remain approximately linear, below 50ms, until competition level 64, when they start to rise, with the 99th percentile accelerating rapidly from competition level 128.

Measurement 1: RSFN packaging with protocol 3

flow x competition (STS performance)
latency vs. concurrency (STS performance)

Measurement 2: RSFN unpacking with protocol 3

flow x competition (STS performance)
latency vs. concurrency (STS performance)

METHODOLOGY:

Architecture

We use AWS EC2 with an isolated VPC and three machines connected to this VPC in the following configuration.

Two machines with Windows Server 2019 and STS Server 7.0.3.

A machine with Ubuntu 22.04 and STS Client Java 7.0.3.

Both STS Servers with one domain configured for the RSFN 3 protocol containing an ISPB with a version key , and the parameter set to 1000.

STS Client Java configuration file edited for the two servers above with the same 50% weight.

Private key in PKCS file 12

Measurement 1: RSFN 3 packaging

In this measurement, we evaluated the performance of STS 7.0.3 in the RSFN 3 packaging operation using the tool .

Tool logging configuration file set to level .

Data file generated with a size of 1 KiB.

Each round of testing was carried out with:

  1. restart both STS server services;
  2. run the tool once and discard the result;
  3. run the tool 5 times and collect the result.

The latency percentiles are obtained from the tool's output and the total execution time is obtained from the output of the time tool.

Measurement control:

Measurement 2: unpacking RSFN 3

In this measurement, we evaluated the performance of STS 7.0.3 in the RSFN 3 packaging operation using the tool .

Tool logging configuration file set to level .

Data file generated with a size of 1 KiB.

Each round of testing was carried out with:

  1. restart both STS server services;
  2. run the tool once and discard the result;
  3. run the tool 5 times and collect the result.

The latency percentiles are obtained from the tool's output and the total execution time is obtained from the tool's output .
Measurement control: