Skip to main content

MQTT Publish Benchmark

Benchmark parameters​

  • Broker: mosquitto 2.0.22 (Windows x64)
  • OPC Router: 5.5
  • Hardware: Windows Sandbox (Intel i7-13700, 4 GB RAM)
  • Payload size: 100 bytes
  • MQTT version: 5
  • OPC Router cycle time (expert setting): 5 ms

Benchmark tool:​

https://github.com/dist-sys/mqttloader

  • num_subscribers = 1
  • num_publishers = 0

Evaluation​

Message throughput was measured for 1, 10, 50, and 100 connections (transfer flows), each with QoS levels 0, 1, and 2. The following overview summarizes the average values for QoS 0:

ConnectionsWithout poolWith poolPer connection without pool
164 msg/s–64 msg/s
10640 msg/s–64 msg/s
502,670 msg/s3,025 msg/s53 msg/s
1004,233 msg/s5,556 msg/s42 msg/s

Throughput scales linearly up to 10 connections—each connection contributes about 64 messages per second. Beyond that, the rate per connection decreases: to 53 messages per second at 50 connections, and to 42 messages per second at 100 connections.

The connection pool recovers some of this loss. With a pool size of Min 10 / max 100, throughput increases by about 13% at 50 connections (from 2,670 to 3,025 msg/s) and by about 31% at 100 connections (from 4,233 to 5,556 msg/s). The gain therefore increases with the number of connections; the pool was not measured for 1 and 10 connections.

QoS reduces throughput. With 100 connections and no pool, QoS 1 is about 11% and QoS 2 about 37% lower than QoS 0 (3,784 and 2,664, respectively, compared to 4,233 msg/s). With the pool, the ratio remains on the same order of magnitude.

note

This test series ran in Windows Sandbox with 4 GB of RAM; the other test series in this chapter ran on servers with 16 GB. The figures are therefore not comparable to the other test series; they show the relative performance of the variants to one another, not the performance limit of the OPC router.

Results without connection pool​

1 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 65
Average throughput [msg/s]: 64.283
Number of received messages: 3857
Per second throughput [msg/s]: 64, 64, 65, 64, 64, 64, 65, 64, 64, 63, 64, 65, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 63, 65, 64, 64, 64, 64, 64, 65, 64, 65, 64, 65, 64, 65, 64, 64, 65, 64, 65, 65, 65, 64, 64, 65, 65, 65, 65, 64, 64, 65, 64, 64, 65, 65, 64, 64

QoS 1​

Maximum throughput [msg/s]: 65
Average throughput [msg/s]: 63.098
Number of received messages: 3849
Per second throughput [msg/s]: 64, 64, 65, 65, 64, 64, 65, 64, 65, 65, 64, 64, 64, 65, 64, 64, 64, 65, 63, 64, 64, 64, 64, 64, 64, 64, 65, 64, 64, 65, 64, 63, 64, 64, 64, 64, 65, 64, 65, 64, 63, 65, 63, 65, 64, 64, 64, 64, 65, 64, 64, 63, 64, 64, 64, 64, 64, 63, 64, 64, 1

QoS 2​

Maximum throughput [msg/s]: 66
Average throughput [msg/s]: 64.133
Number of received messages: 3848
Per second throughput [msg/s]: 65, 65, 64, 64, 64, 64, 64, 64, 65, 64, 64, 64, 66, 63, 64, 64, 64, 64, 63, 64, 65, 64, 64, 64, 65, 64, 64, 64, 65, 64, 64, 64, 66, 64, 64, 63, 64, 64, 65, 65, 64, 64, 64, 64, 64, 64, 64, 64, 63, 64, 64, 64, 65, 64, 64, 64, 63, 64, 64, 64

10 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 650
Average throughput [msg/s]: 640.083
Number of received messages: 38405
Per second throughput [msg/s]: 640, 640, 640, 640, 640, 640, 640, 640, 640, 640, 640, 640, 634, 640, 640, 644, 640, 641, 640, 640, 641, 647, 620, 640, 649, 640, 640, 640, 640, 650, 640, 640, 640, 640, 644, 636, 650, 630, 640, 640, 640, 630, 640, 640, 640, 640, 650, 640, 640, 640, 640, 640, 640, 640, 640, 639, 640, 640, 640, 640

QoS 1​

Maximum throughput [msg/s]: 648
Average throughput [msg/s]: 616.426
Number of received messages: 37602
Per second throughput [msg/s]: 630, 615, 645, 616, 613, 616, 648, 639, 623, 640, 618, 622, 626, 634, 626, 629, 642, 639, 601, 637, 589, 626, 630, 638, 627, 635, 632, 620, 635, 626, 624, 618, 634, 628, 639, 623, 608, 624, 624, 617, 629, 622, 627, 633, 630, 634, 625, 627, 632, 635, 620, 614, 638, 634, 621, 636, 635, 587, 631, 626, 10

QoS 2​

Maximum throughput [msg/s]: 615
Average throughput [msg/s]: 565.233
Number of received messages: 33914
Per second throughput [msg/s]: 575, 599, 559, 572, 514, 551, 559, 562, 589, 562, 565, 567, 550, 532, 568, 589, 613, 574, 564, 559, 553, 574, 547, 567, 582, 577, 546, 577, 590, 546, 539, 583, 577, 548, 523, 589, 597, 543, 615, 540, 593, 520, 564, 534, 558, 578, 590, 598, 556, 594, 603, 544, 529, 589, 552, 560, 560, 530, 560, 566

50 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 3117
Average throughput [msg/s]: 2670.117
Number of received messages: 160207
Per second throughput [msg/s]: 2530, 2718, 3051, 2544, 2782, 2552, 2601, 2736, 2728, 2274, 2412, 2397, 3069, 2968, 2752, 2367, 2902, 2679, 2800, 2751, 2411, 2667, 2414, 2940, 2734, 2679, 2873, 2631, 2177, 2642, 2630, 2366, 2711, 2660, 2557, 2559, 2815, 2722, 2415, 2501, 2611, 2978, 2809, 3022, 3117, 3008, 2764, 2579, 2666, 2397, 2624, 2859, 2302, 2870, 2475, 2612, 2693, 2436, 2789, 2879

QoS 1​

Maximum throughput [msg/s]: 2659
Average throughput [msg/s]: 2415.600
Number of received messages: 144936
Per second throughput [msg/s]: 2515, 2416, 2441, 2417, 2450, 2231, 2256, 2434, 2440, 2523, 2304, 2453, 2437, 2408, 2305, 2178, 2659, 2555, 2654, 2650, 2432, 2210, 2372, 2414, 2283, 2547, 2499, 2561, 2506, 2494, 2655, 2256, 2309, 2281, 2367, 2633, 2319, 2493, 2467, 2417, 1990, 2232, 2338, 2227, 2202, 2074, 2524, 2540, 2416, 2615, 2399, 2439, 2440, 2415, 2571, 2570, 2517, 2460, 2359, 2367

QoS 2​

Maximum throughput [msg/s]: 2164
Average throughput [msg/s]: 1895.950
Number of received messages: 113757
Per second throughput [msg/s]: 1874, 1898, 2164, 1806, 1896, 2030, 1952, 2005, 1875, 1676, 2073, 1831, 1878, 1881, 1962, 1962, 1862, 1869, 1986, 1958, 2037, 1687, 1795, 1955, 1558, 1962, 1890, 1856, 1873, 1873, 1713, 2135, 2051, 1911, 1780, 1969, 1802, 1863, 1975, 1979, 1931, 2087, 2044, 1662, 1668, 1854, 1888, 1884, 1786, 1731, 2048, 1866, 1908, 1886, 1912, 1953, 2006, 1910, 1701, 1930

100 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 5286
Average throughput [msg/s]: 4232.721
Number of received messages: 258196
Per second throughput [msg/s]: 4962, 3818, 3976, 3902, 4773, 4216, 4550, 4165, 4658, 4486, 4897, 3660, 5286, 4286, 4257, 4434, 3804, 4358, 4582, 3965, 4051, 4425, 4627, 3948, 3962, 4726, 4880, 4500, 4037, 4627, 3999, 4087, 4331, 5161, 4622, 4045, 3927, 4371, 4300, 4493, 4128, 4292, 3655, 4065, 4442, 4012, 4111, 3646, 4285, 4577, 3966, 4139, 4342, 4086, 4691, 4642, 4129, 4676, 3852, 4240, 66

QoS 1​

Maximum throughput [msg/s]: 4498
Average throughput [msg/s]: 3783.525
Number of received messages: 230795
Per second throughput [msg/s]: 3909, 3787, 3647, 3879, 4232, 3620, 4025, 4127, 3895, 3814, 4022, 4031, 3607, 3673, 3701, 3965, 4459, 3933, 3422, 4021, 3883, 3802, 3801, 3985, 3322, 4143, 4102, 4040, 4237, 4006, 3836, 3930, 3920, 3669, 3837, 3828, 3863, 3694, 3828, 4498, 3675, 3758, 4175, 4321, 3500, 3680, 2033, 2971, 3649, 3941, 4000, 3864, 3899, 4034, 4009, 3801, 3617, 4319, 3837, 3675, 44

QoS 2​

Maximum throughput [msg/s]: 3172
Average throughput [msg/s]: 2664.483
Number of received messages: 159869
Per second throughput [msg/s]: 2482, 2589, 2482, 2309, 2800, 2666, 2762, 2599, 2280, 2836, 2547, 2881, 2786, 2807, 2421, 2845, 2794, 2576, 2922, 2905, 2790, 2531, 2521, 2554, 2757, 2426, 2690, 2745, 2277, 2531, 2474, 2678, 2611, 2410, 2785, 2743, 2364, 2481, 3162, 2812, 2845, 3098, 2526, 2560, 2709, 3172, 2747, 2670, 2677, 2608, 2325, 2616, 2672, 2758, 2704, 2667, 2875, 2560, 2613, 2836

Results with Connection Pool​

Connection pool was set to Min: 10 and Max: 100

50 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 3230
Average throughput [msg/s]: 3024.600
Number of received messages: 181476
Per second throughput [msg/s]: 2933, 3140, 3057, 2997, 3147, 3104, 3047, 3230, 2845, 2997, 3033, 3083, 3000, 3137, 3042, 2877, 2992, 2916, 3092, 3050, 3026, 2901, 3087, 3079, 3023, 3025, 3160, 2918, 3100, 2957, 2971, 3009, 3049, 2898, 3003, 2980, 3156, 3013, 3000, 2968, 3031, 3027, 3023, 2904, 3102, 2991, 2963, 2886, 3019, 3013, 3097, 3091, 2959, 2998, 3175, 3004, 3074, 2977, 3011, 3089

QoS 1​

Maximum throughput [msg/s]: 2890
Average throughput [msg/s]: 2517.607
Number of received messages: 153574
Per second throughput [msg/s]: 2632, 2519, 2852, 2284, 2619, 2526, 2324, 2750, 2440, 2273, 2499, 2595, 2851, 2440, 2537, 2502, 2646, 2611, 2336, 2745, 2890, 2348, 2640, 2292, 2416, 2756, 2363, 2465, 2659, 2366, 2514, 2554, 2483, 2824, 2861, 2467, 2501, 2857, 2518, 2453, 2615, 2453, 2428, 2773, 2420, 2673, 2274, 2845, 2638, 2475, 2510, 2662, 2613, 2630, 2737, 2590, 2467, 2333, 2721, 2455, 54

QoS 2​

Maximum throughput [msg/s]: 2774
Average throughput [msg/s]: 2355.721
Number of received messages: 143699
Per second throughput [msg/s]: 2406, 2272, 2376, 2673, 2289, 2190, 2343, 2607, 2411, 2437, 2157, 2414, 2444, 2693, 2193, 2094, 2551, 2341, 2139, 2105, 2244, 2471, 2268, 2162, 2495, 2620, 2425, 2284, 2442, 2606, 2623, 2362, 2323, 2498, 2629, 2480, 2343, 2311, 2638, 2386, 2360, 2303, 2374, 2734, 2386, 2176, 2426, 2361, 2607, 2446, 2245, 2352, 2341, 2774, 2495, 2183, 2293, 2523, 2262, 2263, 50

100 OPC Router Connections (transfer flows), Timetrigger Cycle 0ms (fast as possible)​

QoS 0​

Maximum throughput [msg/s]: 6118
Average throughput [msg/s]: 5556.230
Number of received messages: 338930
Per second throughput [msg/s]: 5746, 5714, 5629, 5558, 5955, 5545, 5788, 5294, 4577, 5331, 5419, 5866, 5623, 5699, 5864, 5847, 5374, 5583, 5955, 5506, 5211, 5774, 5639, 6001, 5680, 5681, 5249, 5567, 5564, 5918, 5306, 5853, 5517, 5505, 5717, 5925, 5921, 5666, 5388, 5709, 5496, 5403, 5997, 5848, 6118, 5294, 6007, 5574, 5985, 5991, 5464, 6012, 5711, 5737, 5516, 5690, 5640, 5402, 5740, 5635, 6

QoS 1​

Maximum throughput [msg/s]: 4741
Average throughput [msg/s]: 4231.533
Number of received messages: 253892
Per second throughput [msg/s]: 4735, 3911, 3956, 4481, 4631, 4540, 4548, 3769, 3903, 3829, 4349, 4133, 3973, 4535, 4302, 3905, 4043, 4015, 4353, 4211, 4391, 3909, 4379, 4616, 3761, 4537, 4242, 3902, 4018, 4405, 3939, 4400, 3952, 4223, 3664, 4072, 4103, 4582, 4088, 4569, 4270, 4595, 4052, 4207, 4199, 3940, 3946, 4741, 4116, 4000, 4416, 4517, 4094, 4341, 4687, 4577, 4055, 4422, 4376, 4467

QoS 2​

Maximum throughput [msg/s]: 5090
Average throughput [msg/s]: 3677.557
Number of received messages: 224331
Per second throughput [msg/s]: 3863, 3489, 4634, 2873, 4884, 3806, 3844, 3513, 3068, 3456, 4227, 3471, 3013, 4340, 3546, 3734, 3845, 3800, 3067, 3080, 4453, 3952, 3606, 3546, 3697, 2809, 5000, 3543, 3831, 3766, 3444, 4796, 3680, 3189, 3921, 4339, 4280, 2840, 3457, 3466, 4582, 2583, 4902, 3256, 2953, 4613, 3884, 3592, 2858, 3000, 5090, 4149, 3870, 4191, 3710, 3773, 3478, 4477, 3320, 2879, 3