A unified ML software stack within the PyTorch platform for edge devices. It defines new compiler entry points as well as a state-of-art runtime.
Compared to the legacy Lite Interpreter, there are some major benefits:
- Performance wins compared to Lite Interpreter
- Faster (orders of magnitude lower framework tax in both DSP and CPU)
- Much smaller binary size, 1.5 MB vs 30 KB without operators.
- Smaller memory footprint because we do ahead of time memory planning in ExecuTorch and also have clear granular control over where the runtime allocations are done.
- Long term alignment with the direction of PyTorch infrastructure
- Lite Interpreter relies on TorchScript, which is being phased out; ExecuTorch is the planned replacement for Lite Interpreter.
- Model Authoring & Productivity gains
- More and better defined entry points to perform model, device, and/or use-case specific optimizations (e.g. better backend delegation, user-defined compiler transformations, default or user-defined memory planning, etc)
- Ability to lower constructs like dynamic control flow to run on device.
- Minimal binary size (< 50KB not including kernels)
- Minimal framework tax: loading program, initializing executor, kernel and backend-delegate dispatch, runtime memory utilization
- Portable (cross-compile across many toolchains)
- Executes ATen kernels (or ATen custom kernels)
- Executes custom op kernels
- Supports inter op asynchronous execution
- Supports static memory allocation (heapless)
- Supports custom allocation across memory hierarchies
- Supports control flow needed by models
- Allows selective build of kernels
- Allows backend delegation with lightweight interface
- Basics: Terminology
- Wiki (internal-only)
- Static docs website (internal-only)
- Testing (internal-only)
- Setting up ExecuTorch from GitHub
- (Optional) Building with CMake
- Exporting to Executorch
- Executorch Google Colab
executorch ├── backends # 1st party backend implementations. | ├── xnnpack | ├── vulkan | ├── backend_api.py # TODO move to exir/backend | ├── backend_details.py # TODO move to exir/backend | ├── partioner.py # TODO move to exir/backend ├── build # Utilities for managing the build system. ├── bundled_program # Utilities for attaching reference inputs and outputs to models. TODO move to extension ├── codegen # Tooling to autogenerate bindings between kernels and the runtime. TODO move to tool ├── configurations # TODO delete this ├── docs # Static docs tooling ├── examples # Examples of various user flows, such as model export, delegates, and runtime execution. | ├── executor_runner | ├── export | ├── models ├── exir # Ahead of time library, model capture and lowering apis. | ├── capture # Program capture. | ├── dialects # Op sets for various dialects in the export process. | ├── emit # Conversion from ExportedProgram to Executorch execution instructions. | ├── program # Export artifacts. | ├── serialize # Serialize final export artifact. ├── extension # Extensions built on top of the runtime. | ├── aten_util | ├── data_loader # 1st party data loader implementations. | ├── memory_allocator # 1st party memory allocator implementations. | ├── pybindings # Python api for executorch runtime. | ├── pytree # C++ and Python flattening and unflattening lib for pytrees. | ├── testing_util ├── kernels # 1st party kernel implementations. | ├── aten | ├── optimized | ├── portable # Reference implementations of ATen operators. | ├── prim_ops # Special ops used in executorch runtime for control flow and symbolic primitives. | ├── quantized ├── profiler # Utilities for profiling. TODO delete in favor of ETDump in sdk/ ├── runtime # core cpp runtime of executorch | ├── backend # Backend definition and registration. | ├── core # Core structures used across all levels of the runtime | ├── executor # Model loading, initalization, and execution. | ├── kernel # Kernel registration and management. | ├── platform # Layer between architecture specific code and user calls. ├── schema # Executorch program definition, TODO move under serialization/ ├── scripts # Utility scripts for size management, dependency management, etc. ├── sdk # Model profiling, debugging, and introspection: NOT READY YET FOR OSS USE ├── shim # Compatibility layer between OSS and Internal builds ├── test # Broad scoped end2end tests ├── third-party # third-party dependencies ├── util # TODO delete this ExecuTorch is BSD licensed, as found in the LICENSE file.
