Python与云 ——AWS的Python原生应用浅析 张孝峰 亚马逊AWS资深解决方案架构师 Python 30周年 Python发展时间线 2019/10 v3.8 v2.7.17 开始实现 1989/12 v0.9.0 1991/2 v1.0.0 1994/1 v2.0 2000/10 v2.5 2006/9 v2.6 2008/10 v3.0 2008/12 2008/12 v2.7 2010/7 v3.7 2018/6 AWS发展时间线 2019/10 开始实现 1989/12 v0.9.0 1991/2 v1.0.0 1994/1 v2.0 2000/10 v2.5 2006/9 v2.6 2008/10 v3.0 2008/12 v2.7 2010/7 v3.7 2018/6 贝索斯开始思考 微服务 2000 2000 Amazon S3 2006/3 Amazon SQS 2006/7 Amazon EC2 2006/8 开始研发 2004 AWS Lambda 2014/11 2019/10 22个区域 165项服务 Python被称为“瑞士军刀” • Web开发 • AI / 机器学习 • 自动化运维 • 网络爬虫 • 大数据分析 48 82 160 280 516

in Taipei • Focus on Serverless, Container and AWS CDK • Father of a daughter • Speaker in AWS Global Summits, KubeCon and CNCF Webinar • Active in AWS User Groups from Shanghai, Taipei, Korea and Mobile apps Websites Services Internet Amazon CloudFront Amazon CloudWatch monitoring API Gateway cache Any other AWS service All publicly accessible endpoints AWS Lambda functions Endpoints Endpoints in your VPC Regional API Endpoints AWS Lambda functions Endpoints on Amazon EC2 Your VPC AWS Manage APIs with API Gateway Queues Simple Fully-managed Any volume Pub/sub Simple

费良宏 Technical Evangelist | Amazon Web Services lianghon@amazon.com 用 Python 开发 Serverless 应用 @ 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved $whoami • 软件工程师 (1991 – 今) • “无服务”是一种新的模型，它跨越许多不同类别的服务 AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon SQS AWS Step Functions 计 算 数 据 存 储 集 成 Amazon Aurora Serverless Amazon S3 Amazon DynamoDB AWS AppSync 2014 2016.3 2017.12 AWS Lambda Google Cloud Functions Azure Functions Apache OpenWisk Oracle Fn 2017.10 Project Riff “无服务器”计算平台的发展 2017.2 OpenFAAS “Serverless” 好在哪里？ 来源：https://aws.amazon.com/cn/

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 4 Built-in services 153 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore >= 1.4.87 The AWS credentials can be passed as user/password in the

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4 Built-in services 137 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show stored on Amazon S3. • URI scheme: s3 • Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore (Python 2 and Python

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4 Built-in services 125 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore (Python 2 and Python 3) or boto (Python 2 only) The AWS credentials can be passed as

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 4 Built-in services 135 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore The AWS credentials can be passed as user/password in the URI

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4 Built-in services 137 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore The AWS credentials can be passed as user/password in the URI

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 4 Built-in services 141 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore The AWS credentials can be passed as user/password in the URI

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 4 Built-in services 143 4.1 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for web scraping, it can also be used to extract data using APIs (such as Amazon Associates Web Services) or as a general purpose web crawler. 2.1.1 Walk-through of an example spider In order to show URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore >= 1.4.87 The AWS credentials can be passed as user/password in the