www.jd.com docker 原理与应用实践 张成远 docker 原理与应用实践 Agenda • 容器系统整体架构 • Namespace • CGroup • Device Mapper • Pull Image • Start Container • Stop Container • Docker Image Storage 容器系统整体架构 Namespace 控制任务访问设备 • freezer— 挂起 / 恢复任务 Device Mapper • DM 框架为上层应用提供了丰富的设备映射及 IO 策 略方面的支持 • Docker 存储端实现之一使用 DM - thin provision • 上层通过 dmsetup 工具或 libdevmapper 库使用 Device Mapper Docker storage driver • Docker the same volume • an arbitrary depth of recursive snapshots • metadata is stored on a seperate device from data DM-thin provision example • dd if=/dev/zero of=metadata bs=1024k count=128 • dd

PEERROUTES=yes IPV4_FAILURE_FATAL=no NAME=Internet UUID=b23b1970-690d-4af5-b014-4ac822dfd42c DEVICE=ens160 ONBOOT=yes IPADDR=202.114.193.101 GATEWAY=202.114.193.254 NETMASK=255.255.255.0 centos lvdisplay mkfs.xfs -n ftype=1 /dev/mapper/centos-docker mkdir -p /var/lib/docker mount /dev/mapper/centos-docker /var/lib/docker echo "/dev/mapper/centos-docker /var/lib/docker xfs defaults defaults 0 0" >> /etc/fstab xfs_growfs /dev/mapper/centos-docker df –Th A2. 创建 k8s-node1, k8s-node2, k8s-node3 内存：16GB, 系统盘：40GB，docker 数据盘：40GB，ceph 数据盘：200GB 【注】所有节点(k8s-master, k8s-node1, k8s-node2

docker-engine 使用 yum 安装 执行以下命令安装依赖包： $ sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 鉴于国内网络问题，强烈建议使用国内源，官方源请在注释中查看。 执行下面的命令添加 yum 软件源： $ sudo 对只读权限的分支可以逻辑上进行增量地 修改(不影响只读部分的)。 Docker 目前支持的联合文件系统包括 OverlayFS , AUFS , Btrfs , VFS , ZFS 和 Device Mapper 。 各 Linux 发行版 Docker 推荐使用的存储驱动如下表。 Linux 发行版 Docker 推荐使用的存储驱动 Docker CE on Ubuntu aufs on /dev/mapper/VolGroup-lv_root 50G 5.3G 42G 12% / tmpfs 48G 228K 48G 1% /dev/shm /dev/sda1 485M 40M 420M 9% /boot /dev/mapper/VolGroup-lv_home

DBMS_SCHEDULER 框架异步调用外部shell 脚本，然后由这个shell脚本提交一个Hadoop Map-Reduce 作业。该表函数与映射器 (mapper) 之 间使用 Oracle 高级队列特性进行通信。Hadoop mapper 将数据排入一个公共队列，而表函数则 从该队列中取出数据。由于该表函数能够并行运行，因此使用额外的逻辑来确保仅有一个服 务进程提交外部作业。 3 中存储的数据的一个模板实现。显然可能存在其他的甚至可能更好的实现。 下图是图 2 中原始示意图在技术上更准确、更具体的展示，解释了我们要在何处、如何使用 后文给出的部分实际代码： 图 3. 启动 Mapper 作业并检索数据 第 1 步是确定由谁作为查询协调器。对此我们采用一种将具有相同键值的记录写入表的简单 机制。首个插入胜出，作为此进程的查询协调器 (QC)。请注意，QC 表函数调用同时也承担 bash 脚本。这个 bash 脚本就是图 3 中的启动程 序 (launcher)，它在 Hadoop 集群上启动 mapper 进程（第 3 步）。 5 Oracle 白皮书 — 通过 Oracle 并行处理集成 Hadoop 数据 mapper 进程处理数据，并在第 5 步写入一个队列。在本文的示例中，我们选择了一个在集群 范围内可用的队列。现在，我们

"rhcos-${RHCOS_VERSION}-x86_64-aws.x86_64" \ 3 --virtualization-type hvm \ --root-device-name '/dev/xvda' \ --block-device-mappings 'DeviceName=/dev/xvda,Ebs= {DeleteOnTermination=true,SnapshotId=}' storage: disks: - device: /dev/<device_name> 1 partitions: - label: var start_mib: 2 size_mib: 3 filesystems: - device: /dev/disk/by-partlabel/var "rhcos-${RHCOS_VERSION}-x86_64-aws.x86_64" \ 3 --virtualization-type hvm \ --root-device-name '/dev/xvda' \ --block-device-mappings 'DeviceName=/dev/xvda,Ebs= {DeleteOnTermination=true,SnapshotId=}'

variables (GH23573, GH30959) • Series.map() now accepts collections.abc.Mapping subclasses as a mapper (GH29733) • Added an experimental attrs for storing global metadata about a dataset (GH29062) • 1 >>> df.rename(mapper={0: 1}, index={0: 2}) 0 2 1 pandas 1.0.0 >>> df.rename({0: 1}, index={0: 2}) Traceback (most recent call last): ... TypeError: Cannot specify both 'mapper' and any of 'index' 'index' or 'columns' >>> df.rename(mapper={0: 1}, index={0: 2}) Traceback (most recent call last): ... TypeError: Cannot specify both 'mapper' and any of 'index' or 'columns' You can still change the axis

Fixes • Silenced a warning on some Windows environments about “tput: terminal attributes: No such device or address” when detecting the terminal size. This fix only applies to python 3 (GH16496) • Bug DataFrame.rename() also supports an “axis-style” calling convention, where you specify a single mapper and the axis to apply that mapping to. In [258]: df.rename({'one': 'foo', 'two': 'bar'}, axis='columns'}) sparseness conversions); is a view groupby([by, axis, level, as_index, sort, ...]) Group series using mapper (dict or key function, apply given function to group, return result as series) or by a series of

DataFrame.rename() also supports an “axis-style” calling convention, where you specify a single mapper and the axis to apply that mapping to. In [242]: df.rename({'one': 'foo', 'two': 'bar'}, axis='columns') previous page groupby([by, axis, level, as_index, sort, ...]) Group DataFrame or Series using a mapper or by a Series of columns. gt(other[, level, fill_value, axis]) Greater than of series and other matching indices as other ob- ject. rename([index]) Alter Series index labels or name. rename_axis([mapper, index, columns, axis, . . . ]) Set the name of the axis for the index or columns. reorder_levels(order)

集群，则不要创建使用 ed25519 算法的密钥。相反，创建一个使用 rsa 或 ecdsa 算法的密钥。 2. 查看公共 SSH 密钥： --root-device-name '/dev/xvda' \ --block-device-mappings 'DeviceName=/dev/xvda,Ebs= {DeleteOnTermination=true,SnapshotId=}' "rhcos-${RHCOS_VERSION}-x86_64-aws.x86_64" \ 3 --virtualization-type hvm \ --root-device-name '/dev/xvda' \ --block-device-mappings 'DeviceName=/dev/xvda,Ebs= {DeleteOnTermination=true,SnapshotId=}' disks: - device: /dev/disk/by-id/<device_name> 1 partitions: - label: var start_mib: 2 size_mib: 3 filesystems: - device: /dev/disk/by-partlabel/var

DataFrame.rename() also supports an “axis-style” calling convention, where you specify a single mapper and the axis to apply that mapping to. In [238]: df.rename({'one': 'foo', 'two': 'bar'}, axis='columns') conversions); is a view. groupby(self[, by, axis, level, as_index, ...]) Group DataFrame or Series using a mapper or by a Series of columns. gt(self, other[, level, fill_value, axis]) Return Greater than of series as other ob- ject. rename(self[, index]) Alter Series index labels or name. rename_axis(self[, mapper, index, columns, ...]) Set the name of the axis for the index or columns. reorder_levels(self, order)