Pulsar 驱动的分布式系统架构演进

单体架构的局限性

单体架构在初期具有开发简单、部署便捷的优势，但随着业务规模扩大，系统会面临性能瓶颈、扩展性差、技术栈固化等问题。数据库连接池耗尽、服务启动时间过长、局部故障导致整体不可用是典型痛点。

Pulsar 的核心能力

Apache Pulsar 作为云原生分布式消息系统，提供多租户支持、持久化存储、低延迟和高吞吐特性。其分层架构（计算层与存储层分离）和内置的跨地域复制功能，为分布式改造提供基础设施支撑。

服务拆分与解耦

通过 Pulsar 的 Topic 机制将单体应用拆分为多个微服务。订单服务生成事件到 orders-topic，库存服务通过订阅该 Topic 实现库存扣减。这种发布/订阅模式实现服务间异步通信，降低耦合度。

弹性扩展实现

利用 Pulsar 的 partitioned topic 特性横向扩展消费者。当支付服务处理能力不足时，新增消费者实例即可自动加入消费组，实现负载均衡。分区数可按公式动态调整：

N = TPS × AvgProcessingTime / (ConsumerThroughput × AvailabilityTarget)

其中 TPS 为目标吞吐量，AvailabilityTarget 通常设为 0.95。

事务消息保障

通过 Pulsar 事务 API 实现跨服务数据一致性：

Transaction txn = pulsarClient.newTransaction() .withTransactionTimeout(5, TimeUnit.MINUTES) .build(); producer.newMessage(txn).value("order-created").send(); consumer.acknowledgeAsync(messageId, txn); txn.commit();

该机制确保订单创建与库存更新要么同时成功，要么同时回滚。

流量削峰方案

配置 Pulsar 的速率限制和积压策略应对突发流量：

policies: persistent://tenant/ns/topic: publishRate: publishThrottlingRateInMsg: 1000 publishThrottlingRateInByte: 1048576 backlogQuotaMap: destination_storage: limit: 50G policy: producer_request_hold

监控体系建设

集成 Prometheus 和 Grafana 监控关键指标：

• 生产者/消费者延迟分布
• 积压消息数（backlog）
• 订阅者消费位置差距（markDeletePosition - consumerPosition）

灰度发布策略

使用 Pulsar 的 Key_Shared 订阅模式实现灰度路由。新版本服务订阅时携带 metadata，生产者通过消息 Key 控制流量分配：

producer = client.create_producer( topic='feature-topic', message_routing_mode=MessageRoutingMode.CustomPartition, custom_message_router=FeatureFlagRouter() )

多活容灾部署

配置 Pulsar 集群跨地域复制，结合 Kubernetes 的 Topology Spread Constraints 实现故障自动转移：

bin/pulsar-admin namespaces set-clusters tenant/ns \ --clusters us-west,us-east,eu-central

性能优化技巧

针对金融场景优化：

• 启用 Broker 侧的 TLS 硬件加速
• 调整 BookKeeper 的 JournalSyncData 参数
• 使用 Aerospike 代替 RocksDB 作为缓存存储
• 对持久化 Topic 设置分层存储策略

该架构已在某电商平台实现单集群日均处理 20 亿条消息，P99 延迟控制在 15ms 以内，资源利用率提升 40%。迁移过程建议采用双写方案，逐步将单体数据库依赖替换为事件驱动模式。

https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/5800370
https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/5805748
https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/1715452
https://avg.163.com/topic/detail/5823664
https://avg.163.com/topic/detail/8664685
https://avg.163.com/topic/detail/8664684
https://avg.163.com/topic/detail/8664686
https://avg.163.com/topic/detail/8664683
https://avg.163.com/topic/detail/8664690
https://avg.163.com/topic/detail/8664698
https://avg.163.com/topic/detail/8664691
https://avg.163.com/topic/detail/8664697
https://avg.163.com/topic/detail/8664692
https://avg.163.com/topic/detail/8664699
https://avg.163.com/topic/detail/8664693
https://avg.163.com/topic/detail/8664682
https://avg.163.com/topic/detail/8664700
https://avg.163.com/topic/detail/8664706
https://avg.163.com/topic/detail/8664712
https://avg.163.com/topic/detail/8664701
https://avg.163.com/topic/detail/8664495
https://avg.163.com/topic/detail/8664504
https://avg.163.com/topic/detail/8664704
https://avg.163.com/topic/detail/8664711
https://avg.163.com/topic/detail/8664705
https://avg.163.com/topic/detail/8664710
https://avg.163.com/topic/detail/8664688
https://avg.163.com/topic/detail/8664695
https://avg.163.com/topic/detail/8664631
https://avg.163.com/topic/detail/8664709
https://avg.163.com/topic/detail/8664632
https://avg.163.com/topic/detail/8664625
https://avg.163.com/topic/detail/8664643
https://avg.163.com/topic/detail/8664514
https://avg.163.com/topic/detail/8664655
https://avg.163.com/topic/detail/8664651
https://avg.163.com/topic/detail/8664703
https://avg.163.com/topic/detail/8664623
https://avg.163.com/topic/detail/8664681
https://avg.163.com/topic/detail/8664708
https://avg.163.com/topic/detail/5923960
https://avg.163.com/topic/detail/6126718
https://avg.163.com/topic/detail/8664627
https://avg.163.com/topic/detail/8664642
https://avg.163.com/topic/detail/8664644
https://avg.163.com/topic/detail/8664641
https://avg.163.com/topic/detail/8664656
https://avg.163.com/topic/detail/8664527
https://avg.163.com/topic/detail/8664668
https://avg.163.com/topic/detail/8664663
https://avg.163.com/topic/detail/8664640
https://avg.163.com/topic/detail/8664628
https://avg.163.com/topic/detail/8664667
https://avg.163.com/topic/detail/8664689
https://avg.163.com/topic/detail/8664630
https://avg.163.com/topic/detail/8664696
https://avg.163.com/topic/detail/8663830
https://avg.163.com/topic/detail/8664702
https://avg.163.com/topic/detail/8664654
https://avg.163.com/topic/detail/8664707
https://avg.163.com/topic/detail/8664653
https://avg.163.com/topic/detail/8664542
https://avg.163.com/topic/detail/8664611
https://avg.163.com/topic/detail/8664680
https://avg.163.com/topic/detail/8664674
https://avg.163.com/topic/detail/8664637
https://avg.163.com/topic/detail/8664652
https://avg.163.com/topic/detail/8664679
https://avg.163.com/topic/detail/8664687
https://avg.163.com/topic/detail/8664639
https://avg.163.com/topic/detail/8664694
https://avg.163.com/topic/detail/8663831
https://avg.163.com/topic/detail/8664666
https://avg.163.com/topic/detail/8664626
https://avg.163.com/topic/detail/8664676
https://avg.163.com/topic/detail/8664665
https://avg.163.com/topic/detail/8664624
https://avg.163.com/topic/detail/8664673
https://avg.163.com/topic/detail/8664613
https://avg.163.com/topic/detail/8664649
https://avg.163.com/topic/detail/8664560
https://avg.163.com/topic/detail/8664664
https://avg.163.com/topic/detail/8664677
https://avg.163.com/topic/detail/8664650
https://avg.163.com/topic/detail/8664662
https://avg.163.com/topic/detail/8663832
https://avg.163.com/topic/detail/8664621
https://avg.163.com/topic/detail/8664629
https://avg.163.com/topic/detail/8664636
https://avg.163.com/topic/detail/8664647
https://avg.163.com/topic/detail/8664635
https://avg.163.com/topic/detail/8664615
https://avg.163.com/topic/detail/8664559
https://avg.163.com/topic/detail/8664661
https://avg.163.com/topic/detail/8664648
https://avg.163.com/topic/detail/8664572
https://avg.163.com/topic/detail/8664622
https://avg.163.com/topic/detail/8664634
https://avg.163.com/topic/detail/8664646
https://avg.163.com/topic/detail/8664671
https://avg.163.com/topic/detail/8664633
https://avg.163.com/topic/detail/8664638
https://avg.163.com/topic/detail/8664660
https://avg.163.com/topic/detail/8664584
https://avg.163.com/topic/detail/8664610
https://avg.163.com/topic/detail/8664617
https://avg.163.com/topic/detail/8664570
https://avg.163.com/topic/detail/8664678
https://avg.163.com/topic/detail/8664675
https://avg.163.com/topic/detail/8664659
https://avg.163.com/topic/detail/8664556
https://avg.163.com/topic/detail/8664672
https://avg.163.com/topic/detail/8664571
https://avg.163.com/topic/detail/8664658
https://avg.163.com/topic/detail/8664557
https://avg.163.com/topic/detail/8664670
https://avg.163.com/topic/detail/8664645
https://avg.163.com/topic/detail/8664552
https://avg.163.com/topic/detail/8664657
https://avg.163.com/topic/detail/8664558
https://avg.163.com/topic/detail/8664596
https://avg.163.com/topic/detail/8664620
https://avg.163.com/topic/detail/8664669
https://avg.163.com/topic/detail/8664583
https://avg.163.com/topic/detail/8664612
https://avg.163.com/topic/detail/8664614
https://avg.163.com/topic/detail/8664616
https://avg.163.com/topic/detail/8664582
https://avg.163.com/topic/detail/8664619
https://avg.163.com/topic/detail/8664566
https://avg.163.com/topic/detail/8664495
https://avg.163.com/topic/detail/8664580
https://avg.163.com/topic/detail/8664569
https://avg.163.com/topic/detail/8664606
https://avg.163.com/topic/detail/8664581
https://avg.163.com/topic/detail/8664568
https://avg.163.com/topic/detail/8664595
https://avg.163.com/topic/detail/8664554
https://avg.163.com/topic/detail/8664575
https://avg.163.com/topic/detail/8664591
https://avg.163.com/topic/detail/8664604
https://avg.163.com/topic/detail/8664593
https://avg.163.com/topic/detail/8664550
https://avg.163.com/topic/detail/8664565
https://avg.163.com/topic/detail/8664551
https://avg.163.com/topic/detail/8664504
https://avg.163.com/topic/detail/8664589
https://avg.163.com/topic/detail/8664588
https://avg.163.com/topic/detail/8664562
https://avg.163.com/topic/detail/8664599
https://avg.163.com/topic/detail/8664605
https://avg.163.com/topic/detail/8664567
https://avg.163.com/topic/detail/8664576
https://avg.163.com/topic/detail/8664608
https://avg.163.com/topic/detail/8664549
https://avg.163.com/topic/detail/8664579
https://avg.163.com/topic/detail/8664500
https://avg.163.com/topic/detail/8664555
https://avg.163.com/topic/detail/8664514
https://avg.163.com/topic/detail/8664602
https://avg.163.com/topic/detail/8664498
https://avg.163.com/topic/detail/8664577
https://avg.163.com/topic/detail/8664564
https://avg.163.com/topic/detail/8664511
https://avg.163.com/topic/detail/8664561
https://avg.163.com/topic/detail/8664553
https://avg.163.com/topic/detail/8664563
https://avg.163.com/topic/detail/8664586
https://avg.163.com/topic/detail/8664600
https://avg.163.com/topic/detail/8664592
https://avg.163.com/topic/detail/8664512
https://avg.163.com/topic/detail/8664603
https://avg.163.com/topic/detail/8664527
https://avg.163.com/topic/detail/8664490
https://avg.163.com/topic/detail/8664492
https://avg.163.com/topic/detail/8664578
https://avg.163.com/topic/detail/8664590
https://avg.163.com/topic/detail/8664573
https://avg.163.com/topic/detail/8664523
https://avg.163.com/topic/detail/8664507
https://avg.163.com/topic/detail/8664574
https://avg.163.com/topic/detail/8664587
https://avg.163.com/topic/detail/8664496
https://avg.163.com/topic/detail/8664601
https://avg.163.com/topic/detail/8664510
https://avg.163.com/topic/detail/8664497
https://avg.163.com/topic/detail/8664524
https://avg.163.com/topic/detail/8664536
https://avg.163.com/topic/detail/8664520
https://avg.163.com/topic/detail/8664542
https://avg.163.com/topic/detail/8664624
https://avg.163.com/topic/detail/8664508
https://avg.163.com/topic/detail/8664594
https://avg.163.com/topic/detail/8664598
https://avg.163.com/topic/detail/8664531
https://avg.163.com/topic/detail/8664585
https://avg.163.com/topic/detail/8664607
https://avg.163.com/topic/detail/8664494
https://avg.163.com/topic/detail/8664509
https://avg.163.com/topic/detail/8664521
https://avg.163.com/topic/detail/8664505
https://avg.163.com/topic/detail/8664522
https://avg.163.com/topic/detail/8664546
https://avg.163.com/topic/detail/8664535
https://avg.163.com/topic/detail/8664534
https://avg.163.com/topic/detail/8664529
https://avg.163.com/topic/detail/8664635
https://avg.163.com/topic/detail/8664519
https://avg.163.com/topic/detail/8664543
https://avg.163.com/topic/detail/8664597
https://avg.163.com/topic/detail/8664499
https://avg.163.com/topic/detail/8664533
https://avg.163.com/topic/detail/8664506
https://avg.163.com/topic/detail/8664443
https://avg.163.com/topic/detail/8664454
https://avg.163.com/topic/detail/8664517
https://avg.163.com/topic/detail/8664541
https://avg.163.com/topic/detail/8664528
https://avg.163.com/topic/detail/8664548
https://avg.163.com/topic/detail/8664544
https://avg.163.com/topic/detail/8664648
https://avg.163.com/topic/detail/8664441
https://avg.163.com/topic/detail/8664493
https://avg.163.com/topic/detail/8664503
https://avg.163.com/topic/detail/8664491
https://avg.163.com/topic/detail/8664547
https://avg.163.com/topic/detail/8664518
https://avg.163.com/topic/detail/8664502
https://avg.163.com/topic/detail/8664466
https://avg.163.com/topic/detail/8664513
https://avg.163.com/topic/detail/8664539
https://avg.163.com/topic/detail/8664526
https://avg.163.com/topic/detail/8664440
https://avg.163.com/topic/detail/8664659
https://avg.163.com/topic/detail/8664451
https://avg.163.com/topic/detail/8664672
https://avg.163.com/topic/detail/8664452
https://avg.163.com/topic/detail/8664515
https://avg.163.com/topic/detail/8664530
https://avg.163.com/topic/detail/8664540
https://avg.163.com/topic/detail/8664532
https://avg.163.com/topic/detail/8664545
https://avg.163.com/topic/detail/8664475
https://avg.163.com/topic/detail/8664444
https://avg.163.com/topic/detail/8664538
https://avg.163.com/topic/detail/8664438
https://avg.163.com/topic/detail/8664455
https://avg.163.com/topic/detail/8664460
https://avg.163.com/topic/detail/8664464
https://avg.163.com/topic/detail/8664463
https://avg.163.com/topic/detail/8664477
https://avg.163.com/topic/detail/8664488
https://avg.163.com/topic/detail/8664489
https://avg.163.com/topic/detail/8664501
https://avg.163.com/topic/detail/8664516
https://avg.163.com/topic/detail/8664487
https://avg.163.com/topic/detail/8664525
https://avg.163.com/topic/detail/8664449
https://avg.163.com/topic/detail/8664537
https://avg.163.com/topic/detail/8664471
https://avg.163.com/topic/detail/8664437