examples outside of the learner’s control. Negative examples available or only positive? Good training examples selected by a “benevolent” teacher. “Near miss” examples Learner can query an oracle about about class of an unlabeled example in the environment Learner can construct an arbitrary example and query an oracle for its label Learner can design and run experiments directly in the environment without

异步训练流⽔线和多级存储：提升性能，降低内存成本 � 问题： � Learner线程中参数拉取和参数更新对性能影响⼤ � 内存成为主要资源瓶颈。由于需要等待全部参数 就绪，Parameter Server难以利⽤速度慢的存储 介质 样本读取 样本解析 参数拉 取 训练 参数更新 查询Sparse Table 查询Dense Tensor Reader Learner Worker 返回参数 Request 查询Dense Tensor 更新参数 � 常规训练流⽔线 样本读取 样本解析 参数拉取 参数更新 查询Sparse Table 查询Dense Tensor Reader Learner Worker 返回参数 Request Handler Parameter Server 更新参数 � 异步参数处理流⽔线 参数 预准备 Batch⼊队列 Batch⼊队列