Dev

.vscode/launch.json

@@ -0,0 +1,26 @@
+{
+ // 使用 IntelliSense 了解相关属性。 
+ // 悬停以查看现有属性的描述。
+ // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387
+ "version": "0.2.0",
+ "configurations": [
+ "name": "gdb 启动",
+ "type": "cppdbg",
+ "request": "launch",
+ "program": "/home/nihao/w2T/axera/StackFlow/projects/test_kit_demo3/dist/test_kit_demo3",
+ "args": [],
+ "stopAtEntry": false,
+ "cwd": "${workspaceFolder}",
+ "miDebuggerPath": "/opt/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gdb",
+ "miDebuggerServerAddress": "localhost:1234",
+ "environment": [],
+ "externalConsole": true,
+ "MIMode": "gdb",
+ "setupCommands":[{
+ "description": "Enable pretty-printing for gdb",
+ "text": "-enable-pretty-printing",
+ "ignoreFailures": true
+ }]
+
+ ]
+}

README.md

@@ -1,88 +1,133 @@
-# Module-LLM
-
-<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/4.webp" width="25%">
-
-## Description
-
-**Module LLM** is an integrated offline Large Language Model (LLM) inference module designed for terminal devices that require efficient and intelligent interaction. Whether for smart homes, voice assistants, or industrial control, Module LLM provides a smooth and natural AI experience without relying on the cloud, ensuring privacy and stability. Integrated with the **StackFlow** framework and **Arduino/UiFlow** libraries, smart features can be easily implemented with just a few lines of code.<br>
-Powered by the advanced **AX630C** SoC processor, it integrates a 3.2 TOPs high-efficiency NPU with native support for Transformer models, handling complex AI tasks with ease. Equipped with **4GB LPDDR4** memory (1GB available for user applications, 3GB dedicated to hardware acceleration) and **32GB eMMC** storage, it supports parallel loading and sequential inference of multiple models, ensuring smooth multitasking. The main chip's runtime power consumption of approximately 1.5W, making it highly efficient and suitable for long-term operation.<br>
-It features a built-in microphone, speaker, TF storage card, **USB OTG**, and RGB status light, meeting diverse application needs with support for voice interaction and data transfer. The module offers flexible expansion: the onboard SD card slot supports cold/hot firmware upgrades, and the **UART** communication interface simplifies connection and debugging, ensuring continuous optimization and expansion of module functionality. The USB port supports master-slave auto-switching, serving as both a debugging port and allowing connection to additional USB devices like cameras. Users can purchase the LLM debugging kit to add a 100 Mbps Ethernet port and kernel serial port, using it as an SBC.<br>
-The module is compatible with multiple models and comes pre-installed with the **Qwen2.5-0.5B** language model. It features **KWS** (wake word), **ASR** (speech recognition), **LLM** (large language model), and **TTS** (text-to-speech) functionalities, with support for standalone calls or **pipeline** automatic transfer for convenient development. Future support includes Qwen2.5-1.5B, Llama3.2-1B, and InternVL2-1B models, allowing hot model updates to keep up with community trends and accommodate various complex AI tasks. Vision recognition capabilities include support for CLIP, YoloWorld, and future updates for DepthAnything, SegmentAnything, and other advanced models to enhance intelligent recognition and analysis.<br>
-Plug and play with **M5 hosts**, Module LLM offers an easy-to-use AI interaction experience. Users can quickly integrate it into existing smart devices without complex settings, enabling smart functionality and improving device intelligence. This product is suitable for offline voice assistants, text-to-speech conversion, smart home control, interactive robots, and more.
-
-
-
-## Product Features
-
-- Offline inference, 3.2T@INT8 precision computing power
-- Integrated KWS (wake word), ASR (speech recognition), LLM (large language model), TTS (text-to-speech generation)
-- Multi-model parallel processing
-- Onboard 32GB eMMC storage and 4GB LPDDR4 memory
-- Onboard microphone and speaker
-- Serial communication
-- SD card firmware upgrade
-- Supports ADB debugging
-- RGB indicator light
-- Built-in Ubuntu system
-- Supports OTG functionality
-- Compatible with Arduino/UIFlow
-
-> 
-
-## Applications
-
-- Offline voice assistants
-- Text-to-speech conversion
-- Smart home control
-- Interactive robots
-
-## Specifications
-
-| Specification | Parameter |
-| ---------------- | ------------------------------------------------------------------------------------------- |
-| Processor SoC | AX630C@Dual Cortex A53 1.2 GHz <br> MAX.12.8 TOPS @INT4 and 3.2 TOPS @INT8 |
-| Memory | 4GB LPDDR4 (1GB system memory + 3GB dedicated for hardware acceleration) |
-| Storage | 32GB eMMC5.1 |
-| Communication | Serial communication default baud rate 115200@8N1 (adjustable) |
-| Microphone | MSM421A |
-| Audio Driver | AW8737 |
-| Speaker | 8Ω@1W, Size:2014 cavity speaker |
-| Built-in Units | KWS (wake word), ASR (speech recognition), LLM (large language model), TTS (text-to-speech) |
-| RGB Light | 3x RGB LED@2020 driven by LP5562 (status indication) |
-| Power | Idle: 5V@0.5W, Full load: 5V@1.5W |
-| Button | For entering download mode for firmware upgrade |
-| Upgrade Port | SD card / Type-C port |
-| Working Temp | 0-40°C |
-| Product Size | 54x54x13mm |
-| Packaging Size | 133x95x16mm |
-| Product Weight | 17.4g |
-| Packaging Weight | 32.0g |
-
-
-## Related Links
-
-- [AX630C](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/AX630C.pdf)
-
-## PinMap
-
-| Module LLM | RXD | TXD |
-| ------------ | --- | --- |
-| Core (Basic) | G16 | G17 |
-| Core2 | G13 | G14 |
-| CoreS3 | G18 | G17 |
-
->LLM Module Pin Switching| LLM Module has reserved soldering pads for pin switching. In cases of pin multiplexing conflicts, the PCB trace can be cut and reconnected to other sets of pins.
-
-<img alt="module size" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/03.jpg" width="25%" />
-
-> Taking `CoreS3` as an example, the first column (left green box) is the TX pin for serial communication, where users can choose one out of four options as needed (from top to bottom, the pins are G18, G7, G14, and G10). The default is set to IO18. To switch to a different pin, cut the connection on the solder pad (at the red line) — it’s recommended to use a blade for this — and then connect to one of the three remaining pins below. The second column (right green box) is for RX pin selection, and, as with the TX pin, it also allows a choice of one out of four options.
-
-
-
-## Video
-
-- Module LLM product introduction and example showcase [Module_LLM_Video.mp4](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Module_LLM_Video.mp4)
-
-## AI Benchmark Comparison
-
-<img alt="compare" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Benchmark%E5%AF%B9%E6%AF%94.png" width="100%" />
+# StackFlow
+
+<p align="center"><img src="https://static-cdn.m5stack.com/resource/public/assets/m5logo2022.svg" alt="basic" width="300" height="300"></p>
+
+<p align="center">
+ StackFlow is a simple, fast, and elegant one-stop AI service infrastructure project aimed at embedded developers. Its purpose is to enable Makers and Hackers to quickly obtain powerful AI acceleration capabilities in current embedded devices. StackFlow can infuse a wise soul into various human-machine interaction devices.
+</p>
+
+
+
+## Table of Contents
+
+* [Features](#Features)
+* [Demo](#demo)
+* [SystemRequirements](#SystemRequirements)
+* [Compile](#Compile)
+* [Installation](#Installation)
+* [Upgrade](#Upgrade)
+* [Run](#Run)
+* [Configuration](#Configuration)
+* [Interface](#Interface)
+* [Contribution](#Contribution)
+
+
+## Features
+<!-- ![](doc/assets/network.png) -->
+* Distributed communication architecture. Each unit can operate independently or collaborate with other units.
+* Support for multiple models, including but not limited to speech recognition, speech synthesis, image recognition, natural language processing, and LLM large model assistant inference, etc.
+* Internal data flow. Different units can be configured to work together as needed, avoiding complex data processing workflows.
+* Simple and easy to use. Exchange data through standard JSON to quickly implement AI services.
+* Offline operation. Local AI services can be implemented without the need for an internet connection.
+* Multi-platform support, including but not limited to Module LLM, LLM630 Compute Kit, etc.
+* Flexible configuration. All units can be fully configured with operational parameters, allowing for model swapping and modification of model parameters within the same data flow processing scenario.
+* Simple and easy to use. Developers only need to focus on the model and hardware platform without worrying about the underlying communication and data processing details, enabling quick implementation of AI services.
+* Efficient and stable. Data transmission via ZMQ channels ensures high efficiency, low latency, and strong stability.
+* Open source and free. StackFlow is licensed under the MIT License.
+* Multilingual support. The core unit is implemented in C++ with extreme performance optimization and can be extended to support multiple programming languages. (Requires support for ZMQ programming)
+
+StackFlow is continuously being optimized and iterated. While the framework becomes more comprehensive, more features will be added. Stay tuned.
+
+Main working modes of the StackFlow voice assistant:
+
+After startup, KWS, ASR, LLM, TTS, and AUDIO are configured to work collaboratively. When KWS detects a keyword in the audio obtained from the AUDIO unit, it sends a wake-up signal. At this point, ASR starts working, recognizing the audio data from AUDIO and publishing the results to its output channel. Once LLM receives the text data converted by ASR, it begins reasoning and publishes the results to its output channel. TTS, upon receiving the results from LLM, starts voice synthesis and plays the synthesized audio data according to the configuration.
+
+
+## Demo
+- [StackFlow continuous speech recognition](./projects/llm_framework/README.md)
+- [StackFlow LLM large model awakening dialogue](./projects/llm_framework/README.md)
+- [StackFlow TTS voice synthesis playback](./projects/llm_framework/README.md)
+- [StackFlow yolo visual detection](https://github.com/Abandon-ht/ModuleLLM_Development_Guide/tree/dev/ESP32/cpp)
+- [StackFlow VLM image description](https://github.com/Abandon-ht/ModuleLLM_Development_Guide/tree/dev/ESP32/cpp)
+
+## SystemRequirements ##
+The current AI units of StackFlow are built on the AXERA acceleration platform, with the main chip platforms being ax630c and ax650n. The system requirement is Ubuntu.
+
+## Compile ##
+StackFlow mainly operates on embedded Linux devices. Generally, please perform compilation work on the host Linux device. The compilation toolchain is aarch64-none-linux-gnu.
+```bash
+# Install X86 cross-compilation toolchain
+wget https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/linux/llm/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu.tar.gz
+sudo tar zxvf gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu.tar.gz -C /opt
+
+# Install dependencies
+sudo apt install python3 python3-pip libffi-dev
+pip3 install parse scons requests kconfiglib
+
+# Download StackFlow source code
+git clone https://github.com/m5stack/StackFlow.git
+cd StackFlow
+git submodule update --init
+cd projects/llm_framework
+scons distclean
+
+# Compile. Note: When compiling, you need to be connected to the internet to download source code, binary libraries, and other files. Please ensure a stable network connection.
+scons -j22
+
+# Package the deb file. Note: Due to the large size of LLM model files, packaging the deb file requires a considerable amount of disk space. It is recommended to use disk space of 128GB or more. During packaging, a large number of binary files will be downloaded, so please be aware of data usage.
+cd tools
+python3 llm_pack.py
+```
+
+## Installation ##
+The program and model data of StackFlow are separate. After the program installation is complete, you need to download the model data separately and configure it into the program. The installation involves first installing the program package, and then installing the model package. 
+
+Bare-metal environment installation (execute the following commands on the LLM device):
+```bash
+# First, install the dynamic library dependencies.
+dpkg -i ./lib-llm_1.4-m5stack1_arm64.deb
+# Then, install the llm-sys main unit.
+dpkg -i ./llm-sys_1.4-m5stack1_arm64.deb
+# Install other llm units.
+dpkg -i ./llm-xxx_1.4-m5stack1_arm64.deb
+# Install the model package.
+dpkg -i ./llm-xxx_1.4-m5stack1_arm64.deb
+# Note the installation order of lib-llm_1.4-m5stack1_arm64.deb and llm-sys_1.4-m5stack1_arm64.deb. The installation order of other llm units and model packages is not required.
+```
+
+## Upgrade
+When upgrading, you can either upgrade the AI unit individually or upgrade the entire StackFlow framework. 
+When upgrading a single unit, you can upgrade via an SD card or manually install using the `dpkg` command. It's important to note that for minor version packages, you can install the upgrade package individually, but for major version upgrades, all llm units must be installed completely. 
+Command line upgrade package:
+```bash
+# Install the llm units that need upgrading.
+dpkg -i ./llm-xxx_1.4-m5stack1_arm64.deb
+```
+[Device automatic upgrade installation.](https://docs.m5stack.com/en/guide/llm/llm/image)
+## Run ##
+Relevant AI services will automatically run at startup and can also be manually started via command. 
+Check the running status of the sys unit:
+```bash
+systemctl status llm-sys
+```
+You can refer to the systemd service commands for relevant commands.
+
+## Configuration ##
+StackFlow's configuration is divided into two categories: unit operation parameter configuration and model operation parameter configuration. 
+Both types of configuration files use the JSON format and are located in multiple directories. The directories are as follows:
+```
+/opt/m5stack/data/models/
+/opt/m5stack/share/
+```
+## Interface ##
+StackFlow can be accessed via UART and TCP ports. The default baud rate for the UART port is 115200, and the default port for the TCP port is 10001. Parameters can be modified through configuration files.
+
+## Contribution
+
+* If you like this project, please give it a star first;
+* To report a bug, please go to the [issue page](https://github.com/m5stack/StackFlow/issues);
+* If you want to contribute code, feel free to fork it and then submit a pull request;
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=m5stack/StackFlow&type=Date)](https://star-history.com/#m5stack/StackFlow&Date)