Kafka 성능측정 툴

아래와 같이 카프카 빌트인 툴로 퍼포먼스 TEST 가능

PRODUCER TEST

[root@f6fb6ac1b445 bin]# ./kafka-producer-perf-test.sh --topic test --record-size 1000 --num-records 5000 --
producer-props bootstrap.servers=n1:9092,n2:9092,n3:9092 --throughput 100
501 records sent, 100.2 records/sec (0.10 MB/sec), 2.7 ms avg latency, 335.0 ms max latency.
501 records sent, 100.2 records/sec (0.10 MB/sec), 0.8 ms avg latency, 6.0 ms max latency.
501 records sent, 100.1 records/sec (0.10 MB/sec), 0.6 ms avg latency, 6.0 ms max latency.
500 records sent, 100.0 records/sec (0.10 MB/sec), 0.7 ms avg latency, 7.0 ms max latency.
500 records sent, 100.0 records/sec (0.10 MB/sec), 0.7 ms avg latency, 26.0 ms max latency.
501 records sent, 100.1 records/sec (0.10 MB/sec), 0.6 ms avg latency, 3.0 ms max latency.
501 records sent, 100.1 records/sec (0.10 MB/sec), 0.6 ms avg latency, 6.0 ms max latency.
501 records sent, 100.1 records/sec (0.10 MB/sec), 0.5 ms avg latency, 2.0 ms max latency.
500 records sent, 100.0 records/sec (0.10 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.
5000 records sent, 99.998000 records/sec (0.10 MB/sec), 0.84 ms avg latency, 335.00 ms max latency, 1 ms 50th,
1 ms 95th, 5 ms 99th, 14 ms 99.9th.

PRODUCER TEST 파라미터 설명

topic : 토픽이름
record-size : 테스트 레코드 사이즈(byte)
num-records : 생성할 메시지수
producer-prop : 프로듀서 옵션 처리
bootstrap.servers : 브로커 리스트
throughput : 최대 메시지 처리량 제한(초당 몇개의 메시지)

CONSUMER TEST

[root@f6fb6ac1b445 bin]# ./kafka-consumer-perf-test.sh --topic test --show-detailed-stats --group test_group --
broker-list n1:9092,n2:9092,n3:9092 --reporting-interval 500 --messages 5000
time, threadId, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, rebalance.time.ms, fetch.time.ms,
fetch.MB.sec, fetch.nMsg.sec
2021-01-21 07:49:40:657, 0, 0.0010, 0.0018, 1, 1.9380, 1611215380494, -1611215379978, 0.0000, 0.0000

CONSUMER TEST 파라미터 설명

INPUT

topic : 토픽이름
messages : 소비할 메시지 개수

show-detailed-stats : 통계 지표 리포트

group : consumer group
broker-list : 브로커 리스트

reporting-interval : 리포팅 간격

OUTPUT

data.consumed.in.MB : 컨슈밍한 모든 메세지의 크기
MB.sec : 초당 컨슈밍한 데이터 크기
data.consumed.in.nMsg : 테스트 동안 컨슘한 모든 메세지들의 수

nMsg.sec : 초당 컨슘한 메세지의 수

rebalance.time.ms : *리밸런싱한 시간

*리밸런스 : 컨슈머 그룹안에서 컨슈머들은 메시지를 가져오고 있는 토픽의 파티션에 대해 소유권을 공유하고 있습니다. 컨슈머가 추가되거나 삭제되었을 때 토픽에 대한 소유권이 이동하게 되는데, 이를 리밸런스(rebalance) 라고 합니다.

브로커 수에 따른 퍼포먼스 테스트

브로커 수에 따른 퍼포먼스 테스트

Kafka Broker Cluster를 구성하였을때, Broker 수가 증가함에 따라 Throughput이 얼마나 증가하는지, Kafka Cluster Set의 최대 처리량은 언제 어떤 시점에서 나 타나는지, Log를 기록하는 디스크의 성능에 따라 얼마나 성능이 차이나는지 확인

출처: https://team-platform.tistory.com/6

PRODUCE

브로커 수 :3

INPUT cmd : ./kafka-producer-perf-test.sh --topic test --record-size 1000 --num-records 20000000 --producer-props bootstrap.servers=n1: 9092,n2:9092,n3:9092 --throughput 1000000

OUTPUT : 20000000 records sent, 52996.838739 records/sec (50.54 MB/sec), 616.83 ms avg latency, 8551.00 ms max latency, 564 ms 50th, 772 ms 95th, 1005 ms 99th, 8267 ms 99.9th.

-> Output

[root@f6fb6ac1b445 bin]# ./kafka-producer-perf-test.sh --topic test --record-size 1000 --num-records 20000000 --
producer-props bootstrap.servers=n1:9092,n2:9092,n3:9092 --throughput 1000000
210081 records sent, 42016.2 records/sec (40.07 MB/sec), 651.3 ms avg latency, 994.0 ms max latency.
319344 records sent, 63868.8 records/sec (60.91 MB/sec), 517.1 ms avg latency, 609.0 ms max latency.
302848 records sent, 60569.6 records/sec (57.76 MB/sec), 530.0 ms avg latency, 657.0 ms max latency.
284496 records sent, 56899.2 records/sec (54.26 MB/sec), 582.1 ms avg latency, 953.0 ms max latency.
75024 records sent, 13173.7 records/sec (12.56 MB/sec), 696.9 ms avg latency, 4742.0 ms max latency.
238864 records sent, 42700.0 records/sec (40.72 MB/sec), 1129.2 ms avg latency, 4764.0 ms max latency.
302944 records sent, 60588.8 records/sec (57.78 MB/sec), 704.5 ms avg latency, 2129.0 ms max latency.
290432 records sent, 58086.4 records/sec (55.40 MB/sec), 562.3 ms avg latency, 677.0 ms max latency.
284591 records sent, 56918.2 records/sec (54.28 MB/sec), 577.2 ms avg latency, 1016.0 ms max latency.
282337 records sent, 56467.4 records/sec (53.85 MB/sec), 577.7 ms avg latency, 768.0 ms max latency.
293696 records sent, 58739.2 records/sec (56.02 MB/sec), 558.7 ms avg latency, 608.0 ms max latency.
286000 records sent, 57200.0 records/sec (54.55 MB/sec), 570.9 ms avg latency, 637.0 ms max latency.
38112 records sent, 4395.3 records/sec (4.19 MB/sec), 789.7 ms avg latency, 8548.0 ms max latency.
245024 records sent, 49004.8 records/sec (46.73 MB/sec), 1695.8 ms avg latency, 8551.0 ms max latency.
232912 records sent, 46582.4 records/sec (44.42 MB/sec), 712.9 ms avg latency, 873.0 ms max latency.
293456 records sent, 58691.2 records/sec (55.97 MB/sec), 561.9 ms avg latency, 619.0 ms max latency.
276960 records sent, 55392.0 records/sec (52.83 MB/sec), 586.2 ms avg latency, 662.0 ms max latency.
273344 records sent, 54668.8 records/sec (52.14 MB/sec), 605.7 ms avg latency, 889.0 ms max latency.
288448 records sent, 57689.6 records/sec (55.02 MB/sec), 564.4 ms avg latency, 640.0 ms max latency.
239680 records sent, 47936.0 records/sec (45.72 MB/sec), 677.8 ms avg latency, 857.0 ms max latency.
247472 records sent, 49494.4 records/sec (47.20 MB/sec), 642.1 ms avg latency, 899.0 ms max latency.
294848 records sent, 58969.6 records/sec (56.24 MB/sec), 582.0 ms avg latency, 920.0 ms max latency.
303504 records sent, 60700.8 records/sec (57.89 MB/sec), 533.8 ms avg latency, 641.0 ms max latency.
282044 records sent, 56408.8 records/sec (53.80 MB/sec), 596.0 ms avg latency, 840.0 ms max latency.
274932 records sent, 54986.4 records/sec (52.44 MB/sec), 586.5 ms avg latency, 676.0 ms max latency.
257664 records sent, 42504.8 records/sec (40.54 MB/sec), 573.6 ms avg latency, 2124.0 ms max latency.
244608 records sent, 48921.6 records/sec (46.66 MB/sec), 883.3 ms avg latency, 2217.0 ms max latency.
297136 records sent, 59427.2 records/sec (56.67 MB/sec), 545.7 ms avg latency, 657.0 ms max latency.
259472 records sent, 51894.4 records/sec (49.49 MB/sec), 631.9 ms avg latency, 952.0 ms max latency.
287040 records sent, 57408.0 records/sec (54.75 MB/sec), 575.1 ms avg latency, 668.0 ms max latency.
339136 records sent, 67827.2 records/sec (64.69 MB/sec), 478.8 ms avg latency, 615.0 ms max latency.
273648 records sent, 54729.6 records/sec (52.19 MB/sec), 584.8 ms avg latency, 771.0 ms max latency.
249616 records sent, 30996.6 records/sec (29.56 MB/sec), 576.5 ms avg latency, 4396.0 ms max latency.
272208 records sent, 54441.6 records/sec (51.92 MB/sec), 1048.7 ms avg latency, 4401.0 ms max latency.
259504 records sent, 51900.8 records/sec (49.50 MB/sec), 639.8 ms avg latency, 794.0 ms max latency.
279728 records sent, 55945.6 records/sec (53.35 MB/sec), 566.2 ms avg latency, 886.0 ms max latency.
243953 records sent, 48790.6 records/sec (46.53 MB/sec), 695.9 ms avg latency, 962.0 ms max latency.
287423 records sent, 57484.6 records/sec (54.82 MB/sec), 567.1 ms avg latency, 790.0 ms max latency.
302560 records sent, 60512.0 records/sec (57.71 MB/sec), 543.6 ms avg latency, 678.0 ms max latency.
253296 records sent, 50659.2 records/sec (48.31 MB/sec), 642.0 ms avg latency, 858.0 ms max latency.
285376 records sent, 57075.2 records/sec (54.43 MB/sec), 581.7 ms avg latency, 765.0 ms max latency.
298336 records sent, 59667.2 records/sec (56.90 MB/sec), 551.8 ms avg latency, 595.0 ms max latency.
281600 records sent, 56320.0 records/sec (53.71 MB/sec), 581.7 ms avg latency, 699.0 ms max latency.
264384 records sent, 52876.8 records/sec (50.43 MB/sec), 618.9 ms avg latency, 763.0 ms max latency.
283776 records sent, 56755.2 records/sec (54.13 MB/sec), 564.7 ms avg latency, 708.0 ms max latency.
258256 records sent, 51651.2 records/sec (49.26 MB/sec), 645.9 ms avg latency, 797.0 ms max latency.
292512 records sent, 58502.4 records/sec (55.79 MB/sec), 560.6 ms avg latency, 637.0 ms max latency.
273840 records sent, 54768.0 records/sec (52.23 MB/sec), 601.2 ms avg latency, 841.0 ms max latency.
288512 records sent, 57702.4 records/sec (55.03 MB/sec), 568.3 ms avg latency, 698.0 ms max latency.
294032 records sent, 58806.4 records/sec (56.08 MB/sec), 557.5 ms avg latency, 669.0 ms max latency.
267184 records sent, 53436.8 records/sec (50.96 MB/sec), 610.7 ms avg latency, 753.0 ms max latency.
250000 records sent, 50000.0 records/sec (47.68 MB/sec), 656.4 ms avg latency, 870.0 ms max latency.
289968 records sent, 57993.6 records/sec (55.31 MB/sec), 565.3 ms avg latency, 628.0 ms max latency.
252800 records sent, 50560.0 records/sec (48.22 MB/sec), 636.5 ms avg latency, 781.0 ms max latency.
274016 records sent, 54803.2 records/sec (52.26 MB/sec), 601.8 ms avg latency, 750.0 ms max latency.
236832 records sent, 47366.4 records/sec (45.17 MB/sec), 698.6 ms avg latency, 837.0 ms max latency.
298416 records sent, 59683.2 records/sec (56.92 MB/sec), 550.0 ms avg latency, 592.0 ms max latency.
282368 records sent, 56473.6 records/sec (53.86 MB/sec), 578.2 ms avg latency, 698.0 ms max latency.
283888 records sent, 56777.6 records/sec (54.15 MB/sec), 579.1 ms avg latency, 701.0 ms max latency.
275200 records sent, 55040.0 records/sec (52.49 MB/sec), 592.8 ms avg latency, 774.0 ms max latency.
235872 records sent, 47174.4 records/sec (44.99 MB/sec), 602.9 ms avg latency, 1392.0 ms max latency.
264112 records sent, 52822.4 records/sec (50.38 MB/sec), 693.0 ms avg latency, 1429.0 ms max latency.
298288 records sent, 59657.6 records/sec (56.89 MB/sec), 563.0 ms avg latency, 817.0 ms max latency.
299216 records sent, 59843.2 records/sec (57.07 MB/sec), 552.7 ms avg latency, 645.0 ms max latency.
295840 records sent, 59168.0 records/sec (56.43 MB/sec), 544.4 ms avg latency, 705.0 ms max latency.
289600 records sent, 57920.0 records/sec (55.24 MB/sec), 567.9 ms avg latency, 637.0 ms max latency.
286272 records sent, 57254.4 records/sec (54.60 MB/sec), 577.1 ms avg latency, 789.0 ms max latency.
307984 records sent, 61596.8 records/sec (58.74 MB/sec), 526.0 ms avg latency, 562.0 ms max latency.
302768 records sent, 60553.6 records/sec (57.75 MB/sec), 543.5 ms avg latency, 581.0 ms max latency.
292528 records sent, 58505.6 records/sec (55.80 MB/sec), 558.6 ms avg latency, 707.0 ms max latency.
286032 records sent, 57206.4 records/sec (54.56 MB/sec), 573.5 ms avg latency, 773.0 ms max latency.
305872 records sent, 61174.4 records/sec (58.34 MB/sec), 535.8 ms avg latency, 578.0 ms max latency.
284848 records sent, 56969.6 records/sec (54.33 MB/sec), 570.4 ms avg latency, 675.0 ms max latency.
20000000 records sent, 52996.838739 records/sec (50.54 MB/sec), 616.83 ms avg latency, 8551.00 ms max latency,
564 ms 50th, 772 ms 95th, 1005 ms 99th, 8267 ms 99.9th.

CONSUME

브로커 수 :3

INPUT cmd : ./kafka-consumer-perf-test.sh --topic test --show-detailed-stats --group test_group --broker-list n1:9092,n2:9092,n3:9092 -- reporting-interval 500 --messages 1000000

-> OUTPUT

PRODUCE

브로커 수 :2 (컨테이너 하나 중지)

INPUT cmd : ./kafka-producer-perf-test.sh --topic test --record-size 1000 --num-records 20000000 --producer-props bootstrap.servers=n1: 9092,n2:9092,n3:9092 --throughput 1000000

OUTPUT : 20000000 records sent, 60902.267087 records/sec (58.08 MB/sec), 536.70 ms avg latency, 10245.00 ms max latency, 478 ms 50th, 682 ms 95th, 1593 ms 99th, 10108 ms 99.9th.

[root@f6fb6ac1b445 bin]# ./kafka-consumer-perf-test.sh --topic test --show-detailed-stats --group test_group --
broker-list n1:9092,n2:9092,n3:9092 --reporting-interval 500 --messages 1000000
time, threadId, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, rebalance.time.ms, fetch.time.ms,
fetch.MB.sec, fetch.nMsg.sec
[2021-01-21 09:47:20,304] WARN Couldn't resolve server n1:9092 from bootstrap.servers as DNS resolution failed
for n1 (org.apache.kafka.clients.ClientUtils)
2021-01-21 09:47:21:035, 0, 0.0010, 0.0015, 1, 1.5748, 1611222440739, -1611222440104, 0.0000, 0.0000
2021-01-21 09:47:21:541, 0, 46.8721, 92.6308, 49149, 97130.4348, 0, 506, 92.6308, 97130.4348
2021-01-21 09:47:22:045, 0, 118.1612, 141.4466, 123901, 148317.4603, 0, 504, 141.4466, 148317.4603
2021-01-21 09:47:22:557, 0, 181.6378, 123.9777, 190461, 130000.0000, 0, 512, 123.9777, 130000.0000
2021-01-21 09:47:23:060, 0, 229.3062, 94.7683, 240445, 99371.7694, 0, 503, 94.7683, 99371.7694
2021-01-21 09:47:23:566, 0, 274.2281, 88.7784, 287549, 93090.9091, 0, 506, 88.7784, 93090.9091
2021-01-21 09:47:24:069, 0, 352.3531, 155.3181, 369469, 162862.8231, 0, 503, 155.3181, 162862.8231
2021-01-21 09:47:24:589, 0, 402.1578, 95.7782, 421693, 100430.7692, 0, 520, 95.7782, 100430.7692
2021-01-21 09:47:25:129, 0, 454.8922, 97.6563, 476989, 102400.0000, 0, 540, 97.6563, 102400.0000
2021-01-21 09:47:25:636, 0, 500.7677, 90.4843, 525093, 94879.6844, 0, 507, 90.4843, 94879.6844
2021-01-21 09:47:26:148, 0, 554.5015, 104.9489, 581437, 110046.8750, 0, 512, 104.9489, 110046.8750
2021-01-21 09:47:26:648, 0, 593.5640, 78.1250, 622397, 81920.0000, 0, 500, 78.1250, 81920.0000
2021-01-21 09:47:27:161, 0, 648.2515, 106.6033, 679741, 111781.6764, 0, 513, 106.6033, 111781.6764
2021-01-21 09:47:27:663, 0, 701.9625, 106.9939, 736061, 112191.2351, 0, 502, 106.9939, 112191.2351
2021-01-21 09:47:28:170, 0, 764.4625, 123.2742, 801597, 129262.3274, 0, 507, 123.2742, 129262.3274
2021-01-21 09:47:28:718, 0, 832.8218, 124.7434, 873277, 130802.9197, 0, 548, 124.7434, 130802.9197
2021-01-21 09:47:29:309, 0, 872.8609, 67.7480, 915261, 71038.9171, 0, 591, 67.7480, 71038.9171
2021-01-21 09:47:29:820, 0, 889.4625, 32.4884, 932669, 34066.5362, 0, 511, 32.4884, 34066.5362
2021-01-21 09:47:30:350, 0, 930.4781, 77.3880, 975677, 81147.1698, 0, 530, 77.3880, 81147.1698
[root@f6fb6ac1b445 bin]# ./kafka-producer-perf-test.sh --topic test --record-size 1000 --num-records 20000000 --
producer-props bootstrap.servers=n1:9092,n2:9092,n3:9092 --throughput 1000000
[2021-01-21 09:48:13,999] WARN Couldn't resolve server n1:9092 from bootstrap.servers as DNS resolution failed
for n1 (org.apache.kafka.clients.ClientUtils)
90545 records sent, 18109.0 records/sec (17.27 MB/sec), 1393.9 ms avg latency, 2135.0 ms max latency.
197024 records sent, 39404.8 records/sec (37.58 MB/sec), 865.6 ms avg latency, 1149.0 ms max latency.
269856 records sent, 53971.2 records/sec (51.47 MB/sec), 622.4 ms avg latency, 904.0 ms max latency.
332416 records sent, 66483.2 records/sec (63.40 MB/sec), 491.2 ms avg latency, 681.0 ms max latency.
340544 records sent, 68108.8 records/sec (64.95 MB/sec), 483.3 ms avg latency, 540.0 ms max latency.
312432 records sent, 62486.4 records/sec (59.59 MB/sec), 521.8 ms avg latency, 931.0 ms max latency.
358096 records sent, 71619.2 records/sec (68.30 MB/sec), 458.5 ms avg latency, 502.0 ms max latency.
364800 records sent, 72960.0 records/sec (69.58 MB/sec), 450.0 ms avg latency, 491.0 ms max latency.
358592 records sent, 71718.4 records/sec (68.40 MB/sec), 456.7 ms avg latency, 789.0 ms max latency.
353344 records sent, 70668.8 records/sec (67.40 MB/sec), 463.2 ms avg latency, 481.0 ms max latency.
412624 records sent, 82524.8 records/sec (78.70 MB/sec), 397.2 ms avg latency, 535.0 ms max latency.
289600 records sent, 57920.0 records/sec (55.24 MB/sec), 556.6 ms avg latency, 878.0 ms max latency.
358352 records sent, 71670.4 records/sec (68.35 MB/sec), 464.1 ms avg latency, 652.0 ms max latency.
322192 records sent, 64438.4 records/sec (61.45 MB/sec), 504.6 ms avg latency, 665.0 ms max latency.
279488 records sent, 55897.6 records/sec (53.31 MB/sec), 582.9 ms avg latency, 863.0 ms max latency.
342336 records sent, 68467.2 records/sec (65.30 MB/sec), 487.9 ms avg latency, 636.0 ms max latency.
352336 records sent, 70467.2 records/sec (67.20 MB/sec), 463.9 ms avg latency, 500.0 ms max latency.
389488 records sent, 77897.6 records/sec (74.29 MB/sec), 422.8 ms avg latency, 847.0 ms max latency.
457888 records sent, 91577.6 records/sec (87.34 MB/sec), 356.1 ms avg latency, 522.0 ms max latency.
377936 records sent, 75587.2 records/sec (72.09 MB/sec), 433.8 ms avg latency, 498.0 ms max latency.
213072 records sent, 16353.7 records/sec (15.60 MB/sec), 523.0 ms avg latency, 10063.0 ms max latency.
340640 records sent, 68128.0 records/sec (64.97 MB/sec), 1408.1 ms avg latency, 10245.0 ms max latency.
342048 records sent, 68409.6 records/sec (65.24 MB/sec), 477.9 ms avg latency, 511.0 ms max latency.
330160 records sent, 66032.0 records/sec (62.97 MB/sec), 480.6 ms avg latency, 706.0 ms max latency.
354368 records sent, 70873.6 records/sec (67.59 MB/sec), 478.0 ms avg latency, 761.0 ms max latency.
420512 records sent, 84102.4 records/sec (80.21 MB/sec), 389.7 ms avg latency, 492.0 ms max latency.
311040 records sent, 62208.0 records/sec (59.33 MB/sec), 520.5 ms avg latency, 758.0 ms max latency.
345776 records sent, 69155.2 records/sec (65.95 MB/sec), 479.3 ms avg latency, 691.0 ms max latency.
53664 records sent, 7449.2 records/sec (7.10 MB/sec), 478.4 ms avg latency, 6898.0 ms max latency.
386944 records sent, 77388.8 records/sec (73.80 MB/sec), 967.9 ms avg latency, 6905.0 ms max latency.
343856 records sent, 68771.2 records/sec (65.59 MB/sec), 475.0 ms avg latency, 803.0 ms max latency.
368576 records sent, 73715.2 records/sec (70.30 MB/sec), 446.1 ms avg latency, 514.0 ms max latency.
336704 records sent, 67340.8 records/sec (64.22 MB/sec), 487.0 ms avg latency, 514.0 ms max latency.
293760 records sent, 58752.0 records/sec (56.03 MB/sec), 559.2 ms avg latency, 772.0 ms max latency.
22880 records sent, 3663.7 records/sec (3.49 MB/sec), 468.7 ms avg latency, 6387.0 ms max latency.
335088 records sent, 67017.6 records/sec (63.91 MB/sec), 1066.5 ms avg latency, 6461.0 ms max latency.
341344 records sent, 68268.8 records/sec (65.11 MB/sec), 479.2 ms avg latency, 538.0 ms max latency.
327744 records sent, 65548.8 records/sec (62.51 MB/sec), 487.7 ms avg latency, 711.0 ms max latency.
351088 records sent, 70217.6 records/sec (66.96 MB/sec), 479.1 ms avg latency, 738.0 ms max latency.
338928 records sent, 67785.6 records/sec (64.65 MB/sec), 487.3 ms avg latency, 622.0 ms max latency.
341568 records sent, 68313.6 records/sec (65.15 MB/sec), 473.9 ms avg latency, 515.0 ms max latency.
324336 records sent, 64867.2 records/sec (61.86 MB/sec), 505.9 ms avg latency, 734.0 ms max latency.
358528 records sent, 71619.7 records/sec (68.30 MB/sec), 460.6 ms avg latency, 509.0 ms max latency.
355024 records sent, 71004.8 records/sec (67.72 MB/sec), 458.4 ms avg latency, 551.0 ms max latency.
351328 records sent, 70265.6 records/sec (67.01 MB/sec), 467.6 ms avg latency, 630.0 ms max latency.
363728 records sent, 72745.6 records/sec (69.38 MB/sec), 445.8 ms avg latency, 562.0 ms max latency.
344640 records sent, 68928.0 records/sec (65.73 MB/sec), 479.7 ms avg latency, 612.0 ms max latency.
318368 records sent, 63673.6 records/sec (60.72 MB/sec), 514.6 ms avg latency, 651.0 ms max latency.
339120 records sent, 67824.0 records/sec (64.68 MB/sec), 483.6 ms avg latency, 519.0 ms max latency.

337568 records sent, 67513.6 records/sec (64.39 MB/sec), 484.2 ms avg latency, 521.0 ms max latency.
233504 records sent, 46700.8 records/sec (44.54 MB/sec), 697.7 ms avg latency, 1647.0 ms max latency.
70848 records sent, 12703.6 records/sec (12.12 MB/sec), 490.9 ms avg latency, 5062.0 ms max latency.
368816 records sent, 73763.2 records/sec (70.35 MB/sec), 851.0 ms avg latency, 5123.0 ms max latency.
330512 records sent, 66102.4 records/sec (63.04 MB/sec), 492.6 ms avg latency, 520.0 ms max latency.
330032 records sent, 66006.4 records/sec (62.95 MB/sec), 498.0 ms avg latency, 674.0 ms max latency.
342320 records sent, 68464.0 records/sec (65.29 MB/sec), 479.6 ms avg latency, 531.0 ms max latency.
326448 records sent, 65289.6 records/sec (62.27 MB/sec), 496.3 ms avg latency, 565.0 ms max latency.
280720 records sent, 53562.3 records/sec (51.08 MB/sec), 563.4 ms avg latency, 959.0 ms max latency.
169360 records sent, 33872.0 records/sec (32.30 MB/sec), 1058.3 ms avg latency, 2781.0 ms max latency.
325872 records sent, 65174.4 records/sec (62.16 MB/sec), 502.2 ms avg latency, 609.0 ms max latency.
334624 records sent, 66924.8 records/sec (63.82 MB/sec), 489.2 ms avg latency, 509.0 ms max latency.
322672 records sent, 64534.4 records/sec (61.54 MB/sec), 513.9 ms avg latency, 654.0 ms max latency.
337584 records sent, 67516.8 records/sec (64.39 MB/sec), 480.8 ms avg latency, 539.0 ms max latency.
20000000 records sent, 60902.267087 records/sec (58.08 MB/sec), 536.70 ms avg latency, 10245.00 ms max latency,
478 ms 50th, 682 ms 95th, 1593 ms 99th, 10108 ms 99.9th.

CONSUME

브로커 수 : 2 (컨테이너 하나 종료)

INPUT cmd : ./kafka-consumer-perf-test.sh --topic test --show-detailed-stats --group test_group --broker-list n1:9092,n2:9092,n3:9092 -- reporting-interval 500 --messages 1000000

OUTPUT : 아래

저작자표시 비영리 변경금지

'일 > kafka' 카테고리의 다른 글

카프카 프로듀서 (0)	2021.09.26
Kafka 리밸런싱 리스너 동작 test (0)	2021.09.04
Kafka consumer 개발 (0)	2021.09.04
Kafka Consumer 개념 :: 컨슈머, 컨슈머그룹, 리밸런싱 (0)	2021.08.28
kafka 요약 (0)	2021.08.19

데이터 엔지니어 blair2의 개발 블로그

Kafka 성능측정 툴

'일 > kafka' 카테고리의 다른 글

티스토리툴바

Kafka 성능측정 툴

'일 > kafka' 카테고리의 다른 글

관련글

티스토리툴바