Go in TiDB 申砾@PingCAP Agenda • About Me • What is TiDB • TiDB Architecture • SQL Layer Internal • Golang in TiDB • Next Step of TiDB About Me • Shen Li (申砾) • Tech Lead of TiDB, VP of Engineering Why Go? • Productivity • Concurrency • Great for network programming • GC • Standard libraries && Tools • Good performance • Quick improvement Go in TiDB • More than 100k lines of Go code and session is hard • Account for large memory allocation • Account for memory consuming operators New in Go1.8 • Better GC • Pauses 50μs • sort.Slice Next Step of TiDB • Switch to gRPC • Document support

Go Practices in TiDB 姚维 PingCAP wink@pingcap.com Agenda ● How to build a stable database ○ Schrodinger-test platform ○ Failpoint injection ○ Goroutine-leak detection ● Optimization ○ Chunk vs interface{} errors.New("mock commit error") // } …. } https://github.com/pingcap/tidb/blob/master/store/tikv/txn.go#L223 The generated gofail code func (txn *tikvTxn) Commit(ctx context.Context) error { if vmockCommitError or: __fp_mockCommitError.BadType(vmockCommitError, "bool"); }; …. } $ gofail enable store/tikv/txn.go Why we need a new failpoint ● Generated code is not readable ● Concurrent testing will use the same

No builds required to add new rules. BUSINESS PROBLEM Our Answer Go Language MongoDB Mgo/Beego If we had a system that could do inser*on. Two Go programs perform all the feed management work WHY GO - DYNAMIC FEED MANAGEMENT The Feed System Is Driven By Data Go supports Data from Offer and Internal Feeds DYNAMIC RULE MANAGEMENT Our Answer Go Language MongoDB Mgo/Beego Building rules is a core piece of the

Go runtime related problems in TiDB production environment About me ● Arthur Mao(毛康力), Senior Engineer@PingCAP ● TiDB core developer (top3 contributor) ● GitBook about golang internals (@tiancaiamao) consider priority ● CPU dense workload could affect IO latency Conclusion Part II - Memory control ● Go Runtime ○ Allocated from OS (mmaped) ○ Managed Memory ■ Should the memory be returned to the OS? ○ memory footprint is abnormal ● The memory available on this node is not too much Description ● The Go Runtime thinks it does not use much memory ● The OS does not release the memory (RSS is high) Investigate

Go � TiDB ����� mongodb lixia@pingcap.com @���_go About me ��� PingCAP �� 15 ���� PingCAP ���� TiDB ��������� review ������� 12 ������� go ���������� 13 ������������������������ ���� ���� ���@���_go Email: lixia@pingcap.com Agenda mongodb introduction gonzo with memory engine gonzo with TiKV engine TiDB and TiKV Transaction Q & A mongodb introduction mongodb introduction

stars 16,000+ commits Marquee adopters 100+ contributors 700+ Slack members History ● Choosing Go ● TCP, Syscalls, GC, Scheduler 2013 2015 2016 First Kubernetes Production launch Last Sunday.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Learning Go . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Google I/O 2014 - Scaling with Go: YouTube’s Vitess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Vitess Roadmap managing a large number of MySQL instances Vitess database drivers Vitess includes compliant JDBC and Go (Golang) database drivers using a native query protocol. Additionally, it implements the MySQL server

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Learning Go . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Google I/O 2014 - Scaling with Go: YouTube’s Vitess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Vitess Roadmap managing a large number of MySQL instances Vitess database drivers Vitess includes compliant JDBC and Go (Golang) database drivers using a native query protocol. Additionally, it implements the MySQL server

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Learning Go . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Google I/O 2014 - Scaling with Go: YouTube’s Vitess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Vitess Roadmap read traffic to individual cells, Vitess currently serves reads only from the local cell. Writes will go cross-cell when necessary, to wherever the master for that shard resides. Execution Plans Vitess

after del" is not included here because it is not considered a conflict. If everbody wants the row to go away, it does. A row-level conflict resolution rule is a pairing of a situation and an action. There have an Access Control List containing entries which allow or deny access to an item based on the effective identity resulting from authentication. • The Zumero server can support multiple authentication thus establishing the effective identity for this request. The effective identity for a request is the pairing of the scheme string and the user name. For this example, the effective identity pair would