Main Web>TWikiUsers>KonstantinSkaburskas>KSAMQTestPerf (2010-04-15, unknown)

Performance tests of KahaDB & AMQ Persistence in AMQ5.3.1 and 5.4.0

Summary

Persistent messages to queus. Producer and consumer are connected at the same time. 500000 msgs. For both versions the same node (vtb-generic-26) was used. Logging affects a lot - INFO level was set.

The tests for 5.4 were started on vtb-generic-63 as 5.4 was pre-installed there, and very poor messages rates (~130 msgs/s) were obtained. First impression was that persistent layer + KahaDB in 5.4 suck... though, later I realized that there seemed to be a problem with IO/network on vtb-generic-63. So, to put broker versions into equal conditions I moved 5.4 to vtb-generic-26 as well. Results are below in the table. Corollary: there is no difference in performance of persistent layer + KahaDB between the versions smile

NB! Though, there is a problem now [Wednesday, April 14 2010] on gridmsg101/102. The persistent messages rates on queues are extremely poor - ~20 msgs/s. Results show hide

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b gridmsg101 -p 6163 -n 100000 /queue/test.testPerf
p: 2186 (283) msgs in 13.363 secs; rate=163.6 (21.2) msgs/sec
p: 2539 (657) msgs in 35.044 secs; rate=72.5 (18.7) msgs/sec
total msgs sent : 2539
total msgs ackd : 657
total time : 35.044
sent min: 72.5, max: 163.6, mean: 118.0
ackd min: 18.7, max: 21.2, mean: 20.0
[root@vtb-generic-39 TestPerf.2]# 

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b gridmsg101 -p 6163 -n 100000 /queue/test.testPerf
Received : CONNECTED
Header   : session: ID:gridmsg101.cern.ch-51605-1271056248217-2:3349107
c: 82 msgs in 3.542 secs; rate=23.1 msgs/sec
c: 178 msgs in 8.543 secs; rate=20.8 msgs/sec
c: 292 msgs in 13.543 secs; rate=21.6 msgs/sec
c: 390 msgs in 18.543 secs; rate=21.0 msgs/sec
c: 482 msgs in 23.544 secs; rate=20.5 msgs/sec
c: 568 msgs in 28.544 secs; rate=19.9 msgs/sec
total msgs recv : 585
total time : 28.544
min: 19.9, max: 23.1, mean: 21.2
[root@vtb-generic-39 TestPerf.2]#

As the KahaDB's configuration on gridmsg101/102 is the default one (i.e., not for "high" through-put) and most of the messages are persistent - they are all sync'ed to the DB. As there is quite considerable number of the messages going through the brokers the enqueue&dispatch suffers from the KahaDB IO bottleneck. To increase the through-put the configuration should be changed to <kahaDB directory="${activemq.base}/data/kahadb" enableJournalDiskSyncs="false" indexWriteBatchSize="10000" indexCacheSize="1000"/>. Probably, a feature request can be submitted to FUSE to implement per-destination persistence layers (each running in dedicated thread). The latter would allow to configure destinations exclusively for durability / through-put. As well as parallel disk IOs could give a performance gain.

\	KahaDB		AMQ Persistence
\	5.3.1	5.4.0	5.4.0
default	P: sent min: 1561.5, max: 1894.9, mean: 1859.9	sent min: 1699.7, max: 1984.2, mean: 1943.5	sent min: 2664.1, max: 3075.5, mean: 3029.9
	P: ackd min: 1203.0, max: 1888.3, mean: 1829.8	ackd min: 1313.0, max: 1976.3, mean: 1910.0	ackd min: 2282.8, max: 3042.7, mean: 2983.0
	C: min: 1048.5, max: 1888.9, mean: 1826.9	min: 1225.2, max: 1975.9, mean: 1906.7	min: 1972.4, max: 3011.7, mean: 2920.7

no disk syncs	P: sent min: 3772.9, max: 5291.3, mean: 5053.1	sent min: 3797.8, max: 5090.0, mean: 4819.8
	P: ackd min: 3772.3, max: 5291.2, mean: 5049.6	ackd min: 3410.6, max: 5063.5, mean: 4768.6
	C: min: 3718.7, max: 5295.4, mean: 5046.1	min: 3087.6, max: 5074.2, mean: 4741.1

no disk syncs + x10 indexes	P: sent min: 4454.1, max: 5512.7, mean: 5278.9	sent min: 4536.2, max: 5459.3, mean: 5315.2
	P: ackd min: 4445.5, max: 5485.7, mean: 5238.8	ackd min: 4189.9, max: 5438.4, mean: 5270.6
	C: min: min: 3718.1, max: 5491.3, mean: 5210.1	min: 3504.9, max: 5457.7, mean: 5268.5

KahaDB.

Default configuration.

<kahaDB directory="${activemq.base}/data/kahadb"/>

AMQ5.3.1

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 7863 (6058) msgs in 5.036 secs; rate=1561.5 (1203.0) msgs/sec
p: 17258 (15509) msgs in 10.193 secs; rate=1693.2 (1521.6) msgs/sec
p: 26607 (24859) msgs in 15.288 secs; rate=1740.4 (1626.1) msgs/sec
p: 36649 (34832) msgs in 20.353 secs; rate=1800.7 (1711.4) msgs/sec
p: 46690 (44894) msgs in 25.378 secs; rate=1839.8 (1769.0) msgs/sec
p: 56039 (54245) msgs in 30.406 secs; rate=1843.0 (1784.0) msgs/sec
p: 65735 (63910) msgs in 35.832 secs; rate=1834.6 (1783.6) msgs/sec
p: 75107 (73302) msgs in 40.832 secs; rate=1839.4 (1795.2) msgs/sec
p: 85149 (83321) msgs in 45.832 secs; rate=1857.9 (1818.0) msgs/sec
p: 94821 (93067) msgs in 50.836 secs; rate=1865.2 (1830.7) msgs/sec
p: 104420 (102713) msgs in 55.951 secs; rate=1866.3 (1835.8) msgs/sec
p: 113570 (111865) msgs in 61.083 secs; rate=1859.3 (1831.4) msgs/sec
p: 122929 (121015) msgs in 66.083 secs; rate=1860.2 (1831.3) msgs/sec
p: 130853 (129121) msgs in 71.110 secs; rate=1840.1 (1815.8) msgs/sec
p: 141020 (139317) msgs in 76.200 secs; rate=1850.6 (1828.3) msgs/sec
p: 150848 (149077) msgs in 81.293 secs; rate=1855.6 (1833.8) msgs/sec
p: 160676 (158924) msgs in 86.321 secs; rate=1861.4 (1841.1) msgs/sec
p: 170843 (169110) msgs in 91.445 secs; rate=1868.3 (1849.3) msgs/sec
p: 180297 (178272) msgs in 96.445 secs; rate=1869.4 (1848.4) msgs/sec
p: 190160 (188428) msgs in 101.503 secs; rate=1873.4 (1856.4) msgs/sec
p: 198632 (196921) msgs in 106.527 secs; rate=1864.6 (1848.6) msgs/sec
p: 208460 (206769) msgs in 111.554 secs; rate=1868.7 (1853.5) msgs/sec
p: 218393 (216528) msgs in 116.554 secs; rate=1873.8 (1857.8) msgs/sec
p: 228455 (226688) msgs in 121.607 secs; rate=1878.6 (1864.1) msgs/sec
p: 238622 (236883) msgs in 126.676 secs; rate=1883.7 (1870.0) msgs/sec
p: 248788 (247033) msgs in 131.794 secs; rate=1887.7 (1874.4) msgs/sec
p: 257261 (255485) msgs in 136.828 secs; rate=1880.2 (1867.2) msgs/sec
p: 267428 (265678) msgs in 141.969 secs; rate=1883.7 (1871.4) msgs/sec
p: 277255 (275490) msgs in 147.072 secs; rate=1885.2 (1873.2) msgs/sec
p: 286198 (284333) msgs in 152.072 secs; rate=1882.0 (1869.7) msgs/sec
p: 295556 (293817) msgs in 157.133 secs; rate=1880.9 (1869.9) msgs/sec
p: 305045 (303243) msgs in 162.176 secs; rate=1881.0 (1869.8) msgs/sec
p: 314534 (312743) msgs in 167.230 secs; rate=1880.8 (1870.1) msgs/sec
p: 323345 (321545) msgs in 172.278 secs; rate=1876.9 (1866.4) msgs/sec
p: 333173 (331393) msgs in 177.362 secs; rate=1878.5 (1868.5) msgs/sec
p: 343396 (341592) msgs in 182.362 secs; rate=1883.1 (1873.2) msgs/sec
p: 353135 (351059) msgs in 187.362 secs; rate=1884.8 (1873.7) msgs/sec
p: 362657 (360849) msgs in 192.419 secs; rate=1884.7 (1875.3) msgs/sec
p: 373162 (371376) msgs in 197.555 secs; rate=1888.9 (1879.9) msgs/sec
p: 381604 (379482) msgs in 202.555 secs; rate=1884.0 (1873.5) msgs/sec
p: 390785 (388997) msgs in 207.694 secs; rate=1881.5 (1872.9) msgs/sec
p: 400274 (398496) msgs in 212.763 secs; rate=1881.3 (1873.0) msgs/sec
p: 409085 (407299) msgs in 217.825 secs; rate=1878.0 (1869.8) msgs/sec
p: 419252 (417497) msgs in 222.854 secs; rate=1881.3 (1873.4) msgs/sec
p: 429419 (427603) msgs in 227.904 secs; rate=1884.2 (1876.2) msgs/sec
p: 439247 (437436) msgs in 232.992 secs; rate=1885.2 (1877.5) msgs/sec
p: 447719 (445954) msgs in 238.105 secs; rate=1880.3 (1872.9) msgs/sec
p: 458225 (456451) msgs in 243.132 secs; rate=1884.7 (1877.4) msgs/sec
p: 468730 (466957) msgs in 248.201 secs; rate=1888.5 (1881.4) msgs/sec
p: 478897 (477105) msgs in 253.276 secs; rate=1890.8 (1883.7) msgs/sec
p: 489064 (487301) msgs in 258.381 secs; rate=1892.8 (1886.0) msgs/sec
p: 499231 (497498) msgs in 263.464 secs; rate=1894.9 (1888.3) msgs/sec
p: 500000 (500000) msgs in 268.755 secs; rate=1860.4 (1860.4) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 268.755
sent min: 1561.5, max: 1894.9, mean: 1859.9
ackd min: 1203.0, max: 1888.3, mean: 1829.8
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-50069-1271181260751-2:0
c: 2363 msgs in 2.254 secs; rate=1048.5 msgs/sec
c: 10285 msgs in 7.258 secs; rate=1417.1 msgs/sec
c: 19912 msgs in 12.258 secs; rate=1624.4 msgs/sec
c: 29056 msgs in 17.259 secs; rate=1683.6 msgs/sec
c: 38944 msgs in 22.259 secs; rate=1749.6 msgs/sec
c: 48966 msgs in 27.259 secs; rate=1796.3 msgs/sec
c: 58212 msgs in 32.260 secs; rate=1804.5 msgs/sec
c: 66556 msgs in 37.260 secs; rate=1786.3 msgs/sec
c: 76511 msgs in 42.260 secs; rate=1810.5 msgs/sec
c: 86598 msgs in 47.261 secs; rate=1832.4 msgs/sec
c: 96182 msgs in 52.261 secs; rate=1840.4 msgs/sec
c: 105481 msgs in 57.261 secs; rate=1842.1 msgs/sec
c: 114358 msgs in 62.261 secs; rate=1836.7 msgs/sec
c: 123541 msgs in 67.262 secs; rate=1836.7 msgs/sec
c: 131716 msgs in 72.262 secs; rate=1822.8 msgs/sec
c: 141819 msgs in 77.262 secs; rate=1835.5 msgs/sec
c: 151213 msgs in 82.263 secs; rate=1838.2 msgs/sec
c: 161143 msgs in 87.263 secs; rate=1846.6 msgs/sec
c: 171030 msgs in 92.263 secs; rate=1853.7 msgs/sec
c: 180256 msgs in 97.264 secs; rate=1853.3 msgs/sec
c: 189974 msgs in 102.264 secs; rate=1857.7 msgs/sec
c: 198625 msgs in 107.264 secs; rate=1851.7 msgs/sec
c: 208398 msgs in 112.265 secs; rate=1856.3 msgs/sec
c: 218347 msgs in 117.265 secs; rate=1862.0 msgs/sec
c: 228367 msgs in 122.265 secs; rate=1867.8 msgs/sec
c: 238322 msgs in 127.266 secs; rate=1872.6 msgs/sec
c: 248330 msgs in 132.266 secs; rate=1877.5 msgs/sec
c: 256618 msgs in 137.266 secs; rate=1869.5 msgs/sec
c: 266586 msgs in 142.267 secs; rate=1873.8 msgs/sec
c: 276238 msgs in 147.267 secs; rate=1875.8 msgs/sec
c: 285056 msgs in 152.267 secs; rate=1872.1 msgs/sec
c: 294404 msgs in 157.268 secs; rate=1872.0 msgs/sec
c: 303725 msgs in 162.268 secs; rate=1871.7 msgs/sec
c: 313166 msgs in 167.268 secs; rate=1872.2 msgs/sec
c: 321843 msgs in 172.269 secs; rate=1868.3 msgs/sec
c: 331552 msgs in 177.269 secs; rate=1870.3 msgs/sec
c: 341693 msgs in 182.269 secs; rate=1874.7 msgs/sec
c: 351244 msgs in 187.270 secs; rate=1875.6 msgs/sec
c: 360848 msgs in 192.270 secs; rate=1876.8 msgs/sec
c: 371146 msgs in 197.270 secs; rate=1881.4 msgs/sec
c: 379311 msgs in 202.270 secs; rate=1875.3 msgs/sec
c: 388551 msgs in 207.271 secs; rate=1874.6 msgs/sec
c: 397877 msgs in 212.271 secs; rate=1874.4 msgs/sec
c: 406554 msgs in 217.272 secs; rate=1871.2 msgs/sec
c: 416682 msgs in 222.272 secs; rate=1874.6 msgs/sec
c: 426791 msgs in 227.272 secs; rate=1877.9 msgs/sec
c: 436376 msgs in 232.272 secs; rate=1878.7 msgs/sec
c: 444618 msgs in 237.273 secs; rate=1873.9 msgs/sec
c: 455056 msgs in 242.273 secs; rate=1878.3 msgs/sec
c: 465440 msgs in 247.273 secs; rate=1882.3 msgs/sec
c: 475441 msgs in 252.274 secs; rate=1884.6 msgs/sec
c: 485423 msgs in 257.274 secs; rate=1886.8 msgs/sec
c: 495406 msgs in 262.274 secs; rate=1888.9 msgs/sec
c: 500000 msgs in 267.275 secs; rate=1870.7 msgs/sec

total msgs recv : 500000
total time : 267.275
min: 1048.5, max: 1888.9, mean: 1826.9
[root@vtb-generic-39 TestPerf.2]#

AMQ5.4.0

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 8634 (6670) msgs in 5.080 secs; rate=1699.7 (1313.0) msgs/sec
p: 18705 (16819) msgs in 10.247 secs; rate=1825.4 (1641.3) msgs/sec
p: 28400 (26526) msgs in 15.379 secs; rate=1846.6 (1724.8) msgs/sec
p: 37749 (35875) msgs in 20.413 secs; rate=1849.2 (1757.4) msgs/sec
p: 47791 (45937) msgs in 25.471 secs; rate=1876.3 (1803.5) msgs/sec
p: 57832 (55909) msgs in 30.550 secs; rate=1893.0 (1830.1) msgs/sec
p: 68220 (66327) msgs in 35.687 secs; rate=1911.6 (1858.6) msgs/sec
p: 77223 (75321) msgs in 40.702 secs; rate=1897.3 (1850.5) msgs/sec
p: 87611 (85740) msgs in 45.740 secs; rate=1915.4 (1874.5) msgs/sec
p: 98345 (96426) msgs in 50.825 secs; rate=1935.0 (1897.2) msgs/sec
p: 108547 (106698) msgs in 55.826 secs; rate=1944.4 (1911.3) msgs/sec
p: 118714 (116805) msgs in 60.905 secs; rate=1949.2 (1917.8) msgs/sec
p: 129219 (127351) msgs in 65.970 secs; rate=1958.8 (1930.4) msgs/sec
p: 138031 (136154) msgs in 71.083 secs; rate=1941.8 (1915.4) msgs/sec
p: 148197 (146350) msgs in 76.093 secs; rate=1947.6 (1923.3) msgs/sec
p: 158364 (156547) msgs in 81.192 secs; rate=1950.5 (1928.1) msgs/sec
p: 168870 (167004) msgs in 86.322 secs; rate=1956.3 (1934.7) msgs/sec
p: 179037 (177201) msgs in 91.346 secs; rate=1960.0 (1939.9) msgs/sec
p: 189204 (187310) msgs in 96.418 secs; rate=1962.3 (1942.7) msgs/sec
p: 199370 (197504) msgs in 101.486 secs; rate=1964.5 (1946.1) msgs/sec
p: 207504 (205657) msgs in 106.607 secs; rate=1946.4 (1929.1) msgs/sec
p: 217671 (215808) msgs in 111.736 secs; rate=1948.1 (1931.4) msgs/sec
p: 228176 (226352) msgs in 116.874 secs; rate=1952.3 (1936.7) msgs/sec
p: 238343 (236463) msgs in 121.876 secs; rate=1955.6 (1940.2) msgs/sec
p: 248510 (246658) msgs in 126.927 secs; rate=1957.9 (1943.3) msgs/sec
p: 258338 (256505) msgs in 131.983 secs; rate=1957.4 (1943.5) msgs/sec
p: 268505 (266615) msgs in 137.027 secs; rate=1959.5 (1945.7) msgs/sec
p: 276977 (275157) msgs in 142.160 secs; rate=1948.3 (1935.5) msgs/sec
p: 287483 (285614) msgs in 147.237 secs; rate=1952.5 (1939.8) msgs/sec
p: 297506 (295462) msgs in 152.237 secs; rate=1954.2 (1940.8) msgs/sec
p: 307510 (305658) msgs in 157.237 secs; rate=1955.7 (1943.9) msgs/sec
p: 318322 (316465) msgs in 162.300 secs; rate=1961.3 (1949.9) msgs/sec
p: 329127 (327009) msgs in 167.300 secs; rate=1967.3 (1954.6) msgs/sec
p: 337978 (336161) msgs in 172.372 secs; rate=1960.8 (1950.2) msgs/sec
p: 348484 (346618) msgs in 177.376 secs; rate=1964.7 (1954.1) msgs/sec
p: 358989 (357164) msgs in 182.487 secs; rate=1967.2 (1957.2) msgs/sec
p: 369834 (367970) msgs in 187.517 secs; rate=1972.3 (1962.3) msgs/sec
p: 380001 (378165) msgs in 192.600 secs; rate=1973.0 (1963.5) msgs/sec
p: 390845 (388973) msgs in 197.728 secs; rate=1976.7 (1967.2) msgs/sec
p: 401690 (399865) msgs in 202.857 secs; rate=1980.2 (1971.2) msgs/sec
p: 410840 (409017) msgs in 207.984 secs; rate=1975.3 (1966.6) msgs/sec
p: 421685 (419823) msgs in 213.062 secs; rate=1979.2 (1970.4) msgs/sec
p: 432434 (430369) msgs in 218.062 secs; rate=1983.1 (1973.6) msgs/sec
p: 442696 (440826) msgs in 223.176 secs; rate=1983.6 (1975.2) msgs/sec
p: 452524 (450674) msgs in 228.209 secs; rate=1982.9 (1974.8) msgs/sec
p: 462423 (460521) msgs in 233.209 secs; rate=1982.9 (1974.7) msgs/sec
p: 472858 (470979) msgs in 238.315 secs; rate=1984.2 (1976.3) msgs/sec
p: 481669 (479782) msgs in 243.317 secs; rate=1979.6 (1971.8) msgs/sec
p: 492145 (489979) msgs in 248.317 secs; rate=1981.9 (1973.2) msgs/sec
p: 500000 (500000) msgs in 257.194 secs; rate=1944.1 (1944.1) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 257.194
sent min: 1699.7, max: 1984.2, mean: 1943.5
ackd min: 1313.0, max: 1976.3, mean: 1910.0
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-35946-1271193426644-3:2
c: 4621 msgs in 3.772 secs; rate=1225.2 msgs/sec
c: 13893 msgs in 8.772 secs; rate=1583.8 msgs/sec
c: 23606 msgs in 13.772 secs; rate=1714.0 msgs/sec
c: 32842 msgs in 18.773 secs; rate=1749.5 msgs/sec
c: 42514 msgs in 23.780 secs; rate=1787.8 msgs/sec
c: 52362 msgs in 28.773 secs; rate=1819.8 msgs/sec
c: 62491 msgs in 33.774 secs; rate=1850.3 msgs/sec
c: 71355 msgs in 38.774 secs; rate=1840.3 msgs/sec
c: 81749 msgs in 43.774 secs; rate=1867.5 msgs/sec
c: 92240 msgs in 48.776 secs; rate=1891.1 msgs/sec
c: 102540 msgs in 53.775 secs; rate=1906.8 msgs/sec
c: 112549 msgs in 58.775 secs; rate=1914.9 msgs/sec
c: 122857 msgs in 63.775 secs; rate=1926.4 msgs/sec
c: 132804 msgs in 68.791 secs; rate=1930.5 msgs/sec
c: 141574 msgs in 73.780 secs; rate=1918.9 msgs/sec
c: 151725 msgs in 78.781 secs; rate=1925.9 msgs/sec
c: 161834 msgs in 83.786 secs; rate=1931.5 msgs/sec
c: 171903 msgs in 88.785 secs; rate=1936.2 msgs/sec
c: 181961 msgs in 93.799 secs; rate=1939.9 msgs/sec
c: 192064 msgs in 98.790 secs; rate=1944.2 msgs/sec
c: 200877 msgs in 103.790 secs; rate=1935.4 msgs/sec
c: 210113 msgs in 108.790 secs; rate=1931.4 msgs/sec
c: 219970 msgs in 113.791 secs; rate=1933.1 msgs/sec
c: 230261 msgs in 118.791 secs; rate=1938.4 msgs/sec
c: 240382 msgs in 123.793 secs; rate=1941.8 msgs/sec
c: 250239 msgs in 128.792 secs; rate=1943.0 msgs/sec
c: 259910 msgs in 133.798 secs; rate=1942.6 msgs/sec
c: 268809 msgs in 138.792 secs; rate=1936.8 msgs/sec
c: 278638 msgs in 143.793 secs; rate=1937.8 msgs/sec
c: 288721 msgs in 148.793 secs; rate=1940.4 msgs/sec
c: 298641 msgs in 153.794 secs; rate=1941.8 msgs/sec
c: 309091 msgs in 158.794 secs; rate=1946.5 msgs/sec
c: 319703 msgs in 163.794 secs; rate=1951.9 msgs/sec
c: 330220 msgs in 168.794 secs; rate=1956.3 msgs/sec
c: 339228 msgs in 173.795 secs; rate=1951.9 msgs/sec
c: 349523 msgs in 178.795 secs; rate=1954.9 msgs/sec
c: 360009 msgs in 183.795 secs; rate=1958.8 msgs/sec
c: 370196 msgs in 188.795 secs; rate=1960.8 msgs/sec
c: 380626 msgs in 193.796 secs; rate=1964.1 msgs/sec
c: 391340 msgs in 198.796 secs; rate=1968.5 msgs/sec
c: 401954 msgs in 203.797 secs; rate=1972.3 msgs/sec
c: 410789 msgs in 208.797 secs; rate=1967.4 msgs/sec
c: 421443 msgs in 213.797 secs; rate=1971.2 msgs/sec
c: 431976 msgs in 218.797 secs; rate=1974.3 msgs/sec
c: 441737 msgs in 223.798 secs; rate=1973.8 msgs/sec
c: 451843 msgs in 228.798 secs; rate=1974.9 msgs/sec
c: 461724 msgs in 233.798 secs; rate=1974.9 msgs/sec
c: 471848 msgs in 238.799 secs; rate=1975.9 msgs/sec
c: 480827 msgs in 243.819 secs; rate=1972.1 msgs/sec
c: 491215 msgs in 248.839 secs; rate=1974.0 msgs/sec
c: 500000 msgs in 253.860 secs; rate=1969.6 msgs/sec

total msgs recv : 500000
total time : 253.860
min: 1225.2, max: 1975.9, mean: 1906.7
[root@vtb-generic-39 TestPerf.2]#

Disabled disk syncs.

<kahaDB directory="${activemq.base}/data/kahadb" enableJournalDiskSyncs="false"/>

AMQ5.3.1

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 18781 (18778) msgs in 4.978 secs; rate=3772.9 (3772.3) msgs/sec
p: 44511 (44505) msgs in 9.979 secs; rate=4460.6 (4460.0) msgs/sec
p: 72748 (72734) msgs in 14.979 secs; rate=4856.7 (4855.8) msgs/sec
p: 101101 (101098) msgs in 19.979 secs; rate=5060.4 (5060.2) msgs/sec
p: 124563 (124561) msgs in 24.979 secs; rate=4986.7 (4986.6) msgs/sec
p: 148950 (148928) msgs in 29.979 secs; rate=4968.4 (4967.7) msgs/sec
p: 175882 (175771) msgs in 34.980 secs; rate=5028.1 (5024.9) msgs/sec
p: 202329 (202305) msgs in 39.980 secs; rate=5060.7 (5060.1) msgs/sec
p: 236986 (234745) msgs in 44.984 secs; rate=5268.3 (5218.5) msgs/sec
p: 263097 (263091) msgs in 49.984 secs; rate=5263.6 (5263.5) msgs/sec
p: 288487 (288027) msgs in 54.984 secs; rate=5246.7 (5238.3) msgs/sec
p: 316796 (316763) msgs in 59.984 secs; rate=5281.3 (5280.7) msgs/sec
p: 343854 (343846) msgs in 64.985 secs; rate=5291.3 (5291.2) msgs/sec
p: 367781 (367780) msgs in 69.985 secs; rate=5255.2 (5255.1) msgs/sec
p: 393197 (392984) msgs in 74.985 secs; rate=5243.7 (5240.8) msgs/sec
p: 419841 (419834) msgs in 79.985 secs; rate=5249.0 (5248.9) msgs/sec
p: 446577 (446559) msgs in 84.985 secs; rate=5254.8 (5254.6) msgs/sec
p: 473429 (473427) msgs in 89.985 secs; rate=5261.2 (5261.2) msgs/sec
p: 499337 (499325) msgs in 94.986 secs; rate=5257.0 (5256.8) msgs/sec
p: 500000 (500000) msgs in 100.112 secs; rate=4994.4 (4994.4) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 100.112
sent min: 3772.9, max: 5291.3, mean: 5053.1
ackd min: 3772.3, max: 5291.2, mean: 5049.6
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-37753-1271182219177-2:4
c: 16359 msgs in 4.399 secs; rate=3718.7 msgs/sec
c: 41747 msgs in 9.399 secs; rate=4441.7 msgs/sec
c: 69883 msgs in 14.399 secs; rate=4853.3 msgs/sec
c: 98420 msgs in 19.401 secs; rate=5072.9 msgs/sec
c: 121935 msgs in 24.401 secs; rate=4997.1 msgs/sec
c: 146048 msgs in 29.400 secs; rate=4967.6 msgs/sec
c: 172242 msgs in 34.400 secs; rate=5007.0 msgs/sec
c: 199357 msgs in 39.401 secs; rate=5059.7 msgs/sec
c: 230659 msgs in 44.405 secs; rate=5194.5 msgs/sec
c: 260174 msgs in 49.401 secs; rate=5266.5 msgs/sec
c: 283722 msgs in 54.402 secs; rate=5215.3 msgs/sec
c: 313611 msgs in 59.402 secs; rate=5279.5 msgs/sec
c: 341034 msgs in 64.402 secs; rate=5295.4 msgs/sec
c: 365421 msgs in 69.403 secs; rate=5265.2 msgs/sec
c: 389199 msgs in 74.403 secs; rate=5230.9 msgs/sec
c: 417043 msgs in 79.403 secs; rate=5252.2 msgs/sec
c: 443792 msgs in 84.404 secs; rate=5258.0 msgs/sec
c: 469980 msgs in 89.404 secs; rate=5256.8 msgs/sec
c: 496535 msgs in 94.404 secs; rate=5259.7 msgs/sec
c: 500000 msgs in 99.405 secs; rate=5029.9 msgs/sec

total msgs recv : 500000
total time : 99.405
min: 3718.7, max: 5295.4, mean: 5046.1
[root@vtb-generic-39 TestPerf.2]#

AMQ5.4.0

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 18958 (17025) msgs in 4.992 secs; rate=3797.8 (3410.6) msgs/sec
p: 43775 (43771) msgs in 9.992 secs; rate=4381.1 (4380.7) msgs/sec
p: 71579 (69721) msgs in 14.993 secs; rate=4774.3 (4650.3) msgs/sec
p: 95013 (92450) msgs in 20.020 secs; rate=4746.0 (4618.0) msgs/sec
p: 118158 (115839) msgs in 25.022 secs; rate=4722.2 (4629.6) msgs/sec
p: 148049 (146908) msgs in 30.024 secs; rate=4931.1 (4893.1) msgs/sec
p: 169771 (168970) msgs in 35.024 secs; rate=4847.2 (4824.4) msgs/sec
p: 195715 (194375) msgs in 40.026 secs; rate=4889.7 (4856.3) msgs/sec
p: 218727 (218725) msgs in 45.026 secs; rate=4857.8 (4857.8) msgs/sec
p: 241751 (241750) msgs in 50.027 secs; rate=4832.4 (4832.3) msgs/sec
p: 264818 (264815) msgs in 55.027 secs; rate=4812.5 (4812.4) msgs/sec
p: 289170 (289166) msgs in 60.028 secs; rate=4817.3 (4817.2) msgs/sec
p: 316231 (313770) msgs in 65.028 secs; rate=4863.0 (4825.2) msgs/sec
p: 344837 (342448) msgs in 70.028 secs; rate=4924.3 (4890.2) msgs/sec
p: 379851 (377346) msgs in 75.148 secs; rate=5054.7 (5021.4) msgs/sec
p: 405928 (403233) msgs in 80.148 secs; rate=5064.7 (5031.1) msgs/sec
p: 431681 (431100) msgs in 85.151 secs; rate=5069.6 (5062.8) msgs/sec
p: 458871 (456367) msgs in 90.151 secs; rate=5090.0 (5062.2) msgs/sec
p: 484072 (481808) msgs in 95.154 secs; rate=5087.3 (5063.5) msgs/sec
p: 500000 (500000) msgs in 103.455 secs; rate=4833.0 (4833.0) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 103.455
sent min: 3797.8, max: 5090.0, mean: 4819.8
ackd min: 3410.6, max: 5063.5, mean: 4768.6
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-60209-1271194620399-3:0
c: 10806 msgs in 3.500 secs; rate=3087.6 msgs/sec
c: 35548 msgs in 8.497 secs; rate=4183.8 msgs/sec
c: 59809 msgs in 13.521 secs; rate=4423.6 msgs/sec
c: 86000 msgs in 18.513 secs; rate=4645.3 msgs/sec
c: 108766 msgs in 23.514 secs; rate=4625.6 msgs/sec
c: 137586 msgs in 28.520 secs; rate=4824.2 msgs/sec
c: 161835 msgs in 33.518 secs; rate=4828.3 msgs/sec
c: 187194 msgs in 38.519 secs; rate=4859.8 msgs/sec
c: 210316 msgs in 43.525 secs; rate=4832.1 msgs/sec
c: 234070 msgs in 48.530 secs; rate=4823.2 msgs/sec
c: 257512 msgs in 53.534 secs; rate=4810.3 msgs/sec
c: 281460 msgs in 58.537 secs; rate=4808.3 msgs/sec
c: 306065 msgs in 63.536 secs; rate=4817.2 msgs/sec
c: 336720 msgs in 68.537 secs; rate=4913.0 msgs/sec
c: 369455 msgs in 73.537 secs; rate=5024.1 msgs/sec
c: 395189 msgs in 78.537 secs; rate=5031.9 msgs/sec
c: 423188 msgs in 83.538 secs; rate=5065.8 msgs/sec
c: 449075 msgs in 88.538 secs; rate=5072.1 msgs/sec
c: 474461 msgs in 93.538 secs; rate=5072.4 msgs/sec
c: 500000 msgs in 98.538 secs; rate=5074.2 msgs/sec

total msgs recv : 500000
total time : 98.538
min: 3087.6, max: 5074.2, mean: 4741.1
[root@vtb-generic-39 TestPerf.2]#

Disabled disk syncs. Increased x10 indexWriteBatchSize and indexCacheSize.

<kahaDB directory="${activemq.base}/data/kahadb" enableJournalDiskSyncs="false" indexWriteBatchSize="10000" indexCacheSize="1000"/>

AMQ5.3.1

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 22175 (22132) msgs in 4.979 secs; rate=4454.1 (4445.5) msgs/sec
p: 49035 (47245) msgs in 9.979 secs; rate=4913.9 (4734.5) msgs/sec
p: 74945 (72883) msgs in 15.024 secs; rate=4988.2 (4851.0) msgs/sec
p: 104739 (102185) msgs in 20.024 secs; rate=5230.5 (5103.0) msgs/sec
p: 130439 (130436) msgs in 25.025 secs; rate=5212.4 (5212.3) msgs/sec
p: 158751 (156701) msgs in 30.025 secs; rate=5287.3 (5219.0) msgs/sec
p: 190238 (188750) msgs in 35.026 secs; rate=5431.4 (5388.9) msgs/sec
p: 218417 (218409) msgs in 40.026 secs; rate=5456.9 (5456.7) msgs/sec
p: 245963 (245494) msgs in 45.029 secs; rate=5462.3 (5451.9) msgs/sec
p: 275216 (272627) msgs in 50.070 secs; rate=5496.6 (5444.9) msgs/sec
p: 303587 (302100) msgs in 55.070 secs; rate=5512.7 (5485.7) msgs/sec
p: 330310 (327513) msgs in 60.070 secs; rate=5498.7 (5452.2) msgs/sec
p: 353889 (353885) msgs in 65.070 secs; rate=5438.5 (5438.5) msgs/sec
p: 376602 (376599) msgs in 70.071 secs; rate=5374.6 (5374.6) msgs/sec
p: 401117 (401107) msgs in 75.071 secs; rate=5343.1 (5343.0) msgs/sec
p: 429222 (428022) msgs in 80.071 secs; rate=5360.5 (5345.5) msgs/sec
p: 456512 (454922) msgs in 85.073 secs; rate=5366.1 (5347.4) msgs/sec
p: 484112 (481528) msgs in 90.074 secs; rate=5374.6 (5345.9) msgs/sec
p: 500000 (500000) msgs in 98.112 secs; rate=5096.2 (5096.2) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 98.112
sent min: 4454.1, max: 5512.7, mean: 5278.9
ackd min: 4445.5, max: 5485.7, mean: 5238.8
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-41279-1271182806068-2:0
c: 14365 msgs in 3.864 secs; rate=3718.1 msgs/sec
c: 41584 msgs in 8.864 secs; rate=4691.4 msgs/sec
c: 68997 msgs in 13.864 secs; rate=4976.6 msgs/sec
c: 95941 msgs in 18.865 secs; rate=5085.8 msgs/sec
c: 123737 msgs in 23.866 secs; rate=5184.8 msgs/sec
c: 149690 msgs in 28.875 secs; rate=5184.1 msgs/sec
c: 180716 msgs in 33.870 secs; rate=5335.7 msgs/sec
c: 211142 msgs in 38.871 secs; rate=5431.9 msgs/sec
c: 237416 msgs in 43.918 secs; rate=5405.9 msgs/sec
c: 267091 msgs in 48.914 secs; rate=5460.4 msgs/sec
c: 296064 msgs in 53.915 secs; rate=5491.3 msgs/sec
c: 323503 msgs in 58.916 secs; rate=5490.9 msgs/sec
c: 347812 msgs in 63.919 secs; rate=5441.5 msgs/sec
c: 371260 msgs in 68.920 secs; rate=5386.8 msgs/sec
c: 393438 msgs in 73.918 secs; rate=5322.6 msgs/sec
c: 421904 msgs in 78.917 secs; rate=5346.2 msgs/sec
c: 449443 msgs in 83.917 secs; rate=5355.8 msgs/sec
c: 476398 msgs in 88.918 secs; rate=5357.7 msgs/sec
c: 500000 msgs in 93.917 secs; rate=5323.8 msgs/sec

total msgs recv : 500000
total time : 93.917
min: 3718.1, max: 5491.3, mean: 5210.1
[root@vtb-generic-39 TestPerf.2]#

AMQ5.4.0

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 22580 (20856) msgs in 4.978 secs; rate=4536.2 (4189.9) msgs/sec
p: 51863 (50865) msgs in 9.978 secs; rate=5197.9 (5097.8) msgs/sec
p: 79931 (77755) msgs in 14.995 secs; rate=5330.4 (5185.3) msgs/sec
p: 106830 (104607) msgs in 19.995 secs; rate=5342.8 (5231.6) msgs/sec
p: 132532 (132524) msgs in 24.995 secs; rate=5302.3 (5302.0) msgs/sec
p: 162503 (162500) msgs in 29.996 secs; rate=5417.6 (5417.5) msgs/sec
p: 190328 (190319) msgs in 34.996 secs; rate=5438.6 (5438.4) msgs/sec
p: 218348 (216556) msgs in 39.996 secs; rate=5459.3 (5414.5) msgs/sec
p: 245403 (242841) msgs in 44.996 secs; rate=5453.9 (5397.0) msgs/sec
p: 268421 (267880) msgs in 49.996 secs; rate=5368.9 (5358.1) msgs/sec
p: 296915 (296902) msgs in 54.996 secs; rate=5398.8 (5398.6) msgs/sec
p: 323305 (323294) msgs in 59.997 secs; rate=5388.7 (5388.5) msgs/sec
p: 349991 (349983) msgs in 64.997 secs; rate=5384.7 (5384.6) msgs/sec
p: 377788 (375788) msgs in 69.997 secs; rate=5397.2 (5368.6) msgs/sec
p: 401010 (400861) msgs in 74.998 secs; rate=5346.9 (5345.0) msgs/sec
p: 424794 (424793) msgs in 79.998 secs; rate=5310.1 (5310.0) msgs/sec
p: 457562 (457552) msgs in 84.998 secs; rate=5383.2 (5383.1) msgs/sec
p: 485540 (485531) msgs in 89.998 secs; rate=5395.0 (5394.9) msgs/sec
p: 500000 (500000) msgs in 97.344 secs; rate=5136.4 (5136.4) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 97.344
sent min: 4536.2, max: 5459.3, mean: 5315.2
ackd min: 4189.9, max: 5438.4, mean: 5270.6
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-45654-1271195155703-3:2
c: 13186 msgs in 3.762 secs; rate=3504.9 msgs/sec
c: 45159 msgs in 8.763 secs; rate=5153.5 msgs/sec
c: 72262 msgs in 13.763 secs; rate=5250.5 msgs/sec
c: 99157 msgs in 18.763 secs; rate=5284.7 msgs/sec
c: 126704 msgs in 23.763 secs; rate=5331.9 msgs/sec
c: 156742 msgs in 28.764 secs; rate=5449.3 msgs/sec
c: 184275 msgs in 33.764 secs; rate=5457.7 msgs/sec
c: 210639 msgs in 38.764 secs; rate=5433.8 msgs/sec
c: 237504 msgs in 43.765 secs; rate=5426.8 msgs/sec
c: 263410 msgs in 48.765 secs; rate=5401.6 msgs/sec
c: 290699 msgs in 53.765 secs; rate=5406.8 msgs/sec
c: 317103 msgs in 58.766 secs; rate=5396.1 msgs/sec
c: 343823 msgs in 63.766 secs; rate=5391.9 msgs/sec
c: 371565 msgs in 68.766 secs; rate=5403.3 msgs/sec
c: 395745 msgs in 73.767 secs; rate=5364.8 msgs/sec
c: 420083 msgs in 78.767 secs; rate=5333.2 msgs/sec
c: 450812 msgs in 83.771 secs; rate=5381.5 msgs/sec
c: 479023 msgs in 88.768 secs; rate=5396.4 msgs/sec
c: 500000 msgs in 93.768 secs; rate=5332.3 msgs/sec

total msgs recv : 500000
total time : 93.768
min: 3504.9, max: 5457.7, mean: 5268.5
[root@vtb-generic-39 TestPerf.2]#

AMQ Persistence

Defaults.

<amqPersistenceAdapter directory="${activemq.base}/data/amq"/>

AMQ5.4.0

PRODUCER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

p: 13273 (11373) msgs in 4.982 secs; rate=2664.1 (2282.8) msgs/sec
p: 29893 (27976) msgs in 10.023 secs; rate=2982.5 (2791.2) msgs/sec
p: 45475 (43560) msgs in 15.067 secs; rate=3018.1 (2891.0) msgs/sec
p: 60710 (58787) msgs in 20.248 secs; rate=2998.3 (2903.4) msgs/sec
p: 76638 (74725) msgs in 25.251 secs; rate=3035.1 (2959.3) msgs/sec
p: 92217 (90042) msgs in 30.253 secs; rate=3048.2 (2976.3) msgs/sec
p: 107636 (105765) msgs in 35.290 secs; rate=3050.1 (2997.0) msgs/sec
p: 123225 (121365) msgs in 40.375 secs; rate=3052.0 (3005.9) msgs/sec
p: 138814 (136963) msgs in 45.386 secs; rate=3058.5 (3017.7) msgs/sec
p: 154742 (152914) msgs in 50.463 secs; rate=3066.5 (3030.2) msgs/sec
p: 170573 (168511) msgs in 55.463 secs; rate=3075.5 (3038.3) msgs/sec
p: 186259 (184373) msgs in 60.849 secs; rate=3061.0 (3030.0) msgs/sec
p: 201170 (199276) msgs in 65.875 secs; rate=3053.8 (3025.1) msgs/sec
p: 216082 (214266) msgs in 70.914 secs; rate=3047.1 (3021.5) msgs/sec
p: 232010 (230173) msgs in 75.963 secs; rate=3054.2 (3030.1) msgs/sec
p: 247483 (245553) msgs in 80.991 secs; rate=3055.7 (3031.8) msgs/sec
p: 263072 (261152) msgs in 86.041 secs; rate=3057.5 (3035.2) msgs/sec
p: 278661 (276752) msgs in 91.067 secs; rate=3060.0 (3039.0) msgs/sec
p: 294250 (292311) msgs in 96.127 secs; rate=3061.1 (3040.9) msgs/sec
p: 309500 (307561) msgs in 101.131 secs; rate=3060.4 (3041.2) msgs/sec
p: 325089 (323115) msgs in 106.194 secs; rate=3061.3 (3042.7) msgs/sec
p: 340001 (338104) msgs in 111.269 secs; rate=3055.7 (3038.6) msgs/sec
p: 354234 (352309) msgs in 116.360 secs; rate=3044.3 (3027.8) msgs/sec
p: 369030 (366863) msgs in 121.360 secs; rate=3040.8 (3022.9) msgs/sec
p: 383718 (381767) msgs in 126.410 secs; rate=3035.5 (3020.1) msgs/sec
p: 399307 (397365) msgs in 131.477 secs; rate=3037.1 (3022.3) msgs/sec
p: 414557 (412616) msgs in 136.537 secs; rate=3036.2 (3022.0) msgs/sec
p: 430146 (428216) msgs in 141.610 secs; rate=3037.5 (3023.9) msgs/sec
p: 445397 (443467) msgs in 146.670 secs; rate=3036.7 (3023.6) msgs/sec
p: 460647 (458718) msgs in 151.726 secs; rate=3036.1 (3023.3) msgs/sec
p: 475558 (473621) msgs in 156.729 secs; rate=3034.3 (3021.9) msgs/sec
p: 490470 (488521) msgs in 161.770 secs; rate=3031.9 (3019.8) msgs/sec
p: 500000 (500000) msgs in 170.024 secs; rate=2940.8 (2940.8) msgs/sec

total msgs sent : 500000
total msgs ackd : 500000
total time : 170.024
sent min: 2664.1, max: 3075.5, mean: 3029.9
ackd min: 2282.8, max: 3042.7, mean: 2983.0
[root@vtb-generic-39 TestPerf.2]#

CONSUMER

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 500000 /queue/test.testPerf

show

hide

Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-44280-1271198290437-3:0
c: 8820 msgs in 4.472 secs; rate=1972.4 msgs/sec
c: 24372 msgs in 9.472 secs; rate=2573.0 msgs/sec
c: 39569 msgs in 14.472 secs; rate=2734.1 msgs/sec
c: 55095 msgs in 19.473 secs; rate=2829.4 msgs/sec
c: 70053 msgs in 24.473 secs; rate=2862.5 msgs/sec
c: 84887 msgs in 29.473 secs; rate=2880.1 msgs/sec
c: 99835 msgs in 34.474 secs; rate=2896.0 msgs/sec
c: 115360 msgs in 39.474 secs; rate=2922.4 msgs/sec
c: 130764 msgs in 44.474 secs; rate=2940.2 msgs/sec
c: 146341 msgs in 49.475 secs; rate=2957.9 msgs/sec
c: 162211 msgs in 54.475 secs; rate=2977.7 msgs/sec
c: 177565 msgs in 59.475 secs; rate=2985.5 msgs/sec
c: 192343 msgs in 64.476 secs; rate=2983.2 msgs/sec
c: 207168 msgs in 69.476 secs; rate=2981.9 msgs/sec
c: 222214 msgs in 74.476 secs; rate=2983.7 msgs/sec
c: 239356 msgs in 79.476 secs; rate=3011.7 msgs/sec
c: 252573 msgs in 84.477 secs; rate=2989.9 msgs/sec
c: 267979 msgs in 89.477 secs; rate=2994.9 msgs/sec
c: 283308 msgs in 94.478 secs; rate=2998.7 msgs/sec
c: 298544 msgs in 99.478 secs; rate=3001.1 msgs/sec
c: 313373 msgs in 104.478 secs; rate=2999.4 msgs/sec
c: 327961 msgs in 109.478 secs; rate=2995.7 msgs/sec
c: 342586 msgs in 114.479 secs; rate=2992.6 msgs/sec
c: 357699 msgs in 119.483 secs; rate=2993.7 msgs/sec
c: 372284 msgs in 124.479 secs; rate=2990.7 msgs/sec
c: 387269 msgs in 129.480 secs; rate=2991.0 msgs/sec
c: 402201 msgs in 134.480 secs; rate=2990.8 msgs/sec
c: 416351 msgs in 139.480 secs; rate=2985.0 msgs/sec
c: 431424 msgs in 144.481 secs; rate=2986.0 msgs/sec
c: 446349 msgs in 149.481 secs; rate=2986.0 msgs/sec
c: 461575 msgs in 154.481 secs; rate=2987.9 msgs/sec
c: 476494 msgs in 159.482 secs; rate=2987.8 msgs/sec
c: 492134 msgs in 164.482 secs; rate=2992.0 msgs/sec
c: 500000 msgs in 169.482 secs; rate=2950.2 msgs/sec

total msgs recv : 500000
total time : 169.482
min: 1972.4, max: 3011.7, mean: 2920.7
[root@vtb-generic-39 TestPerf.2]#

Prefetch size 1 & 1000

Conditions: Clean and dedicated broker (not used by anybody else). 250000 or 550000 persistent msgs into a queue. Default configuration <kahaDB directory="${activemq.base}/data/kahadb"/>

Summary:

on a dedicated broker (only one P and C) the influence of prefetch size on consumer is evident.
for persistent messaging, if both P and C are on-line at the same time - C is completely bound by P's throughput if there are no previously stored messages in the store. Otherwise, yes - increasing prefetch size increases broker's dispatch rate and effectively (if consumer if fast enough) overal consumption throughput.

Three cases.

P is sending, C joins to consume when store has a lot of msgs
1	1000
P: ackd min: 1028.4, max: 1351.1, mean: 1192.1	P: ackd min: 972.4, max: 1327.9, mean: 1215.8
C: min: 461.1, max: 1579.8, mean: 1424.6	C: min: 517.0, max: 2546.8, mean: 2185.4

P and C on-line at the same time
1	1000
P: ackd min: 1231.8, max: 1840.1, mean: 1767.7	P: ackd min: 1267.5, max: 1872.0, mean: 1808.2
C: min: 1170.8, max: 1844.6, mean: 1766.8	C: min: 1211.6, max: 1871.3, mean: 1807.3

P sent & went off-line. C comes on-line to consume
1	1000
P: doesn't matter	P: doesn't matter
C: min: 997.7, max: 3456.8, mean: 2736.6	min: 1226.4, max: 4458.8, mean: 3341.2

P is sending, C joins to consume when store has a lot of msgs

prefetch size 1

prefetch size 1. C was started after P sent ~250000 msgs. (X - seconds, Y - rate). For P the rate is of acknowledged msgs.

prefetch size 1000

prefetch size 1000. C (C yellow) was started after P sent ~250000 msgs. (X - seconds, Y - rate). For P the rate is of acknowledged msgs.

P and C on-line at the same time

For persistent messaging, if both P and C are on-line at the same time - C is bound by P's throughput. Otherwise, yes - increasing prefetch size increases broker's dispatch rate and effectively (if consumer if fast enough) overal consumption throughput.

prefetch size 1

Throughput of C is bound by P.

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
...
total msgs sent : 250000
total msgs ackd : 250000
total time : 139.566
sent min: 1705.7, max: 1854.7, mean: 1827.3
ackd min: 1231.8, max: 1840.1, mean: 1767.7
[root@vtb-generic-39 TestPerf.2]# 

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
Received : CONNECTED
...
total msgs recv : 250000
total time : 139.012
min: 1170.8, max: 1844.6, mean: 1766.8
[root@vtb-generic-39 TestPerf.2]#

prefetch size 1000

Throughput of C is bound by P as there are no extra messages in the store, that C could consume having that big prefetch size.

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
...
total msgs sent : 250000
total msgs ackd : 250000
total time : 137.398
sent min: 1663.6, max: 1889.0, mean: 1864.6
ackd min: 1267.5, max: 1872.0, mean: 1808.2
[root@vtb-generic-39 TestPerf.2]#

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
...
total msgs recv : 250000
total time : 134.568
min: 1211.6, max: 1871.3, mean: 1807.3
[root@vtb-generic-39 TestPerf.2]#

P sent & went off-line. C comes on-line to consume.

P dumps to queue. C goes on-line and gets the messages.

prefetch size 1

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
...
total msgs sent : 250000
total msgs ackd : 250000
total time : 197.880
sent min: 1222.5, max: 1519.2, mean: 1290.2
ackd min: 1088.3, max: 1314.7, mean: 1245.5
[root@vtb-generic-39 TestPerf.2]# 

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-47958-1271315341266-2:13
...
total msgs recv : 250000
total time : 75.058
min: 997.7, max: 3456.8, mean: 2736.6
[root@vtb-generic-39 TestPerf.2]#

prefetch size 1000

P dumps to queue and goes off-line. C goes on-line and gets the messages.

[root@vtb-generic-39 TestPerf.2]# ./msg-test-producer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
...
total msgs sent : 250000
total msgs ackd : 250000
total time : 189.701
sent min: 1317.9, max: 1516.3, mean: 1383.8
ackd min: 1125.0, max: 1369.5, mean: 1338.5
[root@vtb-generic-39 TestPerf.2]# 

[root@vtb-generic-39 TestPerf.2]# ./msg-test-consumer-perf -b vtb-generic-26 -p 6163 -n 250000 /queue/test.testPerf1
Received : CONNECTED
Header   : session: ID:vtb-generic-26.cern.ch-49133-1271317205667-2:2
...
total msgs recv : 250000
total time : 59.989
min: 1226.4, max: 4458.8, mean: 3341.2
[root@vtb-generic-39 TestPerf.2]#

-- KonstantinSkaburskas - 13-Apr-2010

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	prefetchSize1.png	r1	manage	26.0 K	2010-04-15 - 11:09	UnknownUser	prefetch size 1
png	prefetchSize1000.png	r1	manage	22.3 K	2010-04-15 - 11:09	UnknownUser	prefetch size 1000

Topic revision: r5 - 2010-04-15 - unknown

Main

Webs

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
Main All webs

Copyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback