阿里消息中间件的演变历史 双11万亿级数据洪峰的挑战 Apache RocketMQ 未来展望 m w a l i b a b a - i n c . c o m ©2016 Alibaba Middleware Group 阿里消息中间件演变历史 2016 2007 2010 2011 2012 2015 Notify 五彩石项目 交易核心消息流转 Napoli ActiveMQ内核 阿里消息中间件的演变历史 双11万亿级数据洪峰的挑战 Apache RocketMQ 未来展望 m w a l i b a b a - i n c . c o m ©2016 Alibaba Middleware Group n 历年双11消息数量变化 n 消息中间件核心链路 n 低延迟存储 n 容量保障 n 熔断机制 n 多副本高可用 10亿 百亿 千亿 5千亿+ 万亿+ 历年双11消息数量变化 Latency Message Trace MQTT/REST Proxy Relay Service Transaction Message Kafka Service Apache RocketMQ SDK(Java/C++/.Net/PHP/Go) | Console | JMS | HTTP | Schedule Message Order Message File

阿里消息中间件的演变历史 双11万亿级数据洪峰的挑战 Apache RocketMQ 未来展望 m w a l i b a b a - i n c . c o m ©2016 Alibaba Middleware Group 阿里消息中间件演变历史 2016 2007 2010 2011 2012 2015 Notify 五彩石项目 交易核心消息流转 Napoli ActiveMQ内核 阿里消息中间件的演变历史 双11万亿级数据洪峰的挑战 Apache RocketMQ 未来展望 m w a l i b a b a - i n c . c o m ©2016 Alibaba Middleware Group n 历年双11消息数量变化 n 消息中间件核心链路 n 低延迟存储 n 容量保障 n 熔断机制 n 多副本高可用 10亿 百亿 千亿 5千亿+ 万亿+ 历年双11消息数量变化 Latency Message Trace MQTT/REST Proxy Relay Service Transaction Message Kafka Service Apache RocketMQ SDK(Java/C++/.Net/PHP/Go) | Console | JMS | HTTP | Schedule Message Order Message File

designed for B2B Trading and other scenarios MetaQ v1.0 Designed for ordered message, especially for message accumulation Aliware MQ v1.0 Notify v3.0 MetaQ v2.0 v3.0 RocketMQ v3.0 and Non-Ordered message  Broadcast message  Scheduled message  Transactional message  Batched message  One-way message  Traceable message  Retroactive message  Message Filtering  Massive e G r o u p Console Scheduled Message © 2 0 1 7 A l i b a b a M i d d l e w a r e G r o u p Transactional Message – Eventually Consistent © 2 0 1 7 A l i b a b a M i

. 4 4.2 Message Priority ............................................................................................................................................... 4 4.3 Message Order ..... 5 4.4 Message Filter ................................................................................................................................................... 5 4.5 Message Persistence ...................................................................................... 5 4.6 Message Reliablity .....................................................................................

deployment, we could validate the deployment scales, checking the pods and start process queue message, if message length increase, more pods will be pro-actively added - We can also get the number of messages Benchmark the ScaledObject will be deployed when there is change in the scaling as the number of message increases pollingInterval: 30 # Optional. Default: 30 seconds cooldownPeriod: 300 # Optional scaler with metadata, get connection,etc - check if there is message pending from RocketMQ queue to be processed - get queue message - Research all methods for event driven trigger from RocketMQ

setNamesrvAddr("127.0.0.1:9876"); producer.start(); for (int i = 0; i < 9; i++) { try { Message msg = new Message("TopicTest10" ,"TagA" , ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET)); setNamesrvAddr("127.0.0.1:9876"); producer.start(); for (int i = 0; i < 300; i++) { try { Message msg = new Message("TopicTest" ,"TagA" , ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET)); 是消息发送处理器、 PullMessageProcessor 是消息拉取命令处理器。  RequestCode 请求 CODE，用来区分请求的类型，例如 SEND_MESSAGE：表示该请求为消息发 送，PULL_MESSAGE:消息拉取请求。  Pair 用来封装 NettyRequestProcessor 与 ExecuteService 的绑定关系。在 RocketMQ 的网络处理模型中，会为每一个

ge Scheduled Message Batched Message BroadCast Message Messa e Filter Server Triggered Redelivery Message Storage Message Retroactive essage Priority High Availability and Failover Message Track Configuration ordering of messages,and can scale out gracefully Supported Supported, w th sync mode to avoid message loss Supported Supported, property filter expressi ns based on SQL92 Supported High performance

RocketMQ 存储设计 存储⽂文件 - - - - - 1G - - - - - 6M - - - - - 400M 配置和状态相关 消息存储和检索相关 存储设计 producer message(topic, queueId, tags, keys, msgBody) commitlog 8 bytes | 4 bytes | 8 bytes commitLogOffset 如果CC重试不不成功/超时，需要⼿手⼯工⼲干预 Apache RocketMQ 消息事务 Send a message first or execute local transaction first ? Member service Account service Send message Receive message 先执⾏行行本地事务还是先发送消息？ 交易易型分布式事务的 RocketMQ使⽤用场景 Part Storage Part 事务消息存储设计 Half Topic: prepare message TransactionMessageCheckService Check Operation Topic: commit/rollback message Broker Producer with PID Broker到Producer的回查通信 Netty

三：Broker 落地消息 2.1 普通消息落地 Broker 根 据 producer 请 求 的 RequestCode.SEND_MESSAGE 选 择 对 应 的 处 理 器 SendMessageProcessor 根据请求消息内容构建消息内部结构 MessageExtBrokerInner 调 DefaultMessageStore