It might happen, that you run benchmark software and will receive results you did not expect with the hardware of the server you ordered.
In such cases, at first please make sure to checkout the knowledgebase articles about slow CPU / disk performance.
In our case, we usually use network storage in our storage backends, usually CEPH.
CEPH as a scaleable and redundant storage system has some design related disadvantages, especially when it comes to small blocksizes.
The smaller the blocksize, the lower the bandwidth throughput you will see. This can be as bad as < 10 MB/s.
The reason is not in the hardware of your server, but in the fact that is because of the nature of a network storage in combination with a storage system that does checksums and frequent scrubs to ensure the correctness of data. So there is some "organizational" work to be done when saving data. And the amount of organizational work is basically always the same and similar to the so call "overhead" you have when using IP to exchange data.
This leads to situations where you will see very bad benchmark results for small blocksizes and more and more better results the higher the blocksize goes.
Another point is that we are usually using NVMe drives. Those drives do not operate in small blocksizes but in larger ones. So they greatly benefit from larger blocksizes.
And in this larger blocksizes the communication is done in the backend.
That is why under normal circumstances, you will not really feel a bad performance on your server. You will usually not have applications waiting for disk IO time. While with benchmark results of < 10 MB/s normally your system might easily be in trouble and having a laggy performance.
So if you are running benchmarks, you have also to consider the hardware settings and the hardware your server runs on and the fact that checksum'ing network storages behave a lot different from the traditional local storages.
