Leveraging the power of C++ for efficient machine learning on embedded devices Adrian Stanciu adrian.stanciu.pub@gmail.com CppCon, 2023 1 / 50About me ◮ I am a software engineer from Romania ◮ I have Image classification ◮ Hand gesture recognition ◮ Summary ◮ Q&A 4 / 50Motivation 5 / 50Machine Learning (ML) ◮ Subfield of Artificial Inteligence (AI) ◮ Enables computers to learn from data and then consumption ◮ May have real-time performance constraints 7 / 50Machine learning on embedded devices ◮ Alternative to cloud-based machine learning ◮ Advantages: ◮ Real-time processing ◮ Low latency ◮ Reduced bandwidth

micrograd++: A 500 line C++ Machine Learning Library Gautam Sharma Independent Researcher gautamsharma2813@gmail.com Abstract—micrograd++ is a pure C++ machine learning li- brary inspired by Andrej Karpathy’s for building and training machine learning models. By leveraging the performance efficiency of C++, micro- grad++ offers a robust solution for integrating machine learning capabilities directly into C++-based Traditionally, all machine learning libraries are extremely bulky and very hard to integrate as third party dependencies. This aspect scares practitioners to adopt a C++ based machine learning library for prototyping

Canada ● Chair of Programming Languages for Standards Council of Canada Chair of WG21 SG19 Machine Learning Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded ● Editor: C++ SG5 Transactional hardware with built-in reduction operation acceleration. • Work group and subgroup algorithms • Efficient parallel operations between work items • Class template argument deduction (CTAD) and template boilerplate code and streamlines the use of C++ software design patterns • Expanded interoperability • Efficient acceleration by diverse backend acceleration APIs • SYCL atomic operations are now more closely

but have never had a chance to study design patterns and need some resources to help orient them. Learning about design patterns and where to apply them can at the least give you a way to think about how for object-oriented programming 26 ● I really enjoyed this book (as a graphics programmer) for learning design patterns. ○ There’s a free web version here: https://gameprogrammingpatterns.com/ ○ I rendering involves: ■ loading font files, a custom string class, a drawable object class, and a deep hierarchy... ■ Oh, by the way, the text must always be green and uppercase letters.Problem we are

and how to practically apply it well for several years, still learning and enjoying the journey. • I am passionate about energy efficient computing and making the most out of older hardware, researching mathewbensonke Agenda • The talk is an exploration of the importance of C++ as a language for writing efficient code for older hardware which is easier to access in the developing world. • It will also look Technological Improvement, does push down the cost for the “Previous Generations” of Hardware, Efficient Code for Budget/Older Hardware • With a large number of people with less to spend on computer hardware

Canada ● Chair of Programming Languages for Standards Council of Canada Chair of WG21 SG19 Machine Learning Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded ● Editor: C++ SG5 Transactional Critical Advisory Forum • OpenCL/SYCL Safety Critical • Vulkan Safety Critical • JTC1/SC42 Machine Learning WG3 Trustworthiness • ITC22/SC32 SOTIF WG8 SOTIF, WG13, WG14 • SAE ORAD • UL4600 • RISC-V Safety/Security which is much harder in multithread system • Heterogeneous-> AI/ML safety Stage 1: extensive deep analysis of 81 rules • Started in 2019 at a MISRA meeting • Why are there no rules for parallelism

Introduction Conan Documentation, Release 2.3.2 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all previous explanations and code - building on the previous example. This is the recommended approach for learning Conan. • The examples also contain hands-on, fully operational examples with code, aimed to explain created from the same recipe revision. The package transfers, uploads, downloads, will also be more efficient, based on revisions. As long as a given revision exists on the server or in the cache, Conan will

Introduction Conan Documentation, Release 2.5.0 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all previous explanations and code - building on the previous example. This is the recommended approach for learning Conan. • The examples also contain hands-on, fully operational examples with code, aimed to explain created from the same recipe revision. The package transfers, uploads, downloads, will also be more efficient, based on revisions. As long as a given revision exists on the server or in the cache, Conan will

Introduction Conan Documentation, Release 2.4.1 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all previous explanations and code - building on the previous example. This is the recommended approach for learning Conan. • The examples also contain hands-on, fully operational examples with code, aimed to explain created from the same recipe revision. The package transfers, uploads, downloads, will also be more efficient, based on revisions. As long as a given revision exists on the server or in the cache, Conan will