ChatGPT Example

Board	Support Status
ESP32-S3-BOX	YES
ESP32-S3-BOX-Lite	YES
ESP32-S3-BOX-3	YES

In this example, we are utilizing the OpenAI API in conjunction with an ESP-BOX to create a voice-based chatbot. The ESP-BOX is a device or system that incorporates an ESP32-S3 microcontroller. The purpose of this implementation is to enable users to communicate with the chatbot using spoken language. The process involves capturing audio input from the user, sending it to the OpenAI API for processing, and receiving a response that is then converted into speech and played back to the user.

How to use example

• ESP-IDF version master

git checkout master && git pull && git submodule update --init --recursive

• If you encounter any errors during the upgrade process, please make sure to use the ESP-IDF version master at commit ID 53ff7d43dbff642d831a937b066ea0735a6aca24. This specific commit represents a stable state of the ESP-IDF repository and can help ensure a smoother transition or troubleshooting process.

git checkout 53ff7d43dbff642d831a937b066ea0735a6aca24 && git pull && git submodule update --init --recursive

Hardware Required

• A ESP32-S3-BOX，ESP32-S3-BOX-Lite or ESP32-S3-BOX-3
• A USB-C cable for power supply and programming

Build and Flash

There is another project named factory_nvs within the ChatGPT_demo project. It includes the code to store credentials in the NVS (Non-Volatile Storage) of the ESP-Box. On the other hand, Chat_GPT Demo consists of the demo code. Therefore, it is essential to build both projects.

1. Clone the Github repository

git clone https://github.com/espressif/esp-box

2. Change the working directory

cd examples/chatgpt_demo/factory_nvs

3. Hardware Selection

To select the appropriate hardware (ESP32-S3-BOX, ESP32-S3-BOX-Lite or ESP32-S3-BOX-3) and access the HMI Board Cofig, use the following command.

idf.py menuconfig

4. Build the factory_nvs

idf.py build

5. Change the working directory

cd examples/chatgpt_demo/

6. Hardware Selection

To select the appropriate hardware (ESP32-S3-BOX, ESP32-S3-BOX-Lite or ESP32-S3-BOX-3) and access the HMI Board Cofig, use the following command.

idf.py menuconfig

7. Build the chatgpt_demo

idf.py build

8. Flash

idf.py -p PORT flash

Known Issues

1. When encountering compilation errors related to the espressif__esp-sr component, a common solution is to remove the .component_hash file located at managed_components/espressif__esp-sr and proceed with the rebuild. This step helps resolve the issue and allows the compilation process to continue smoothly.
2. If you encounter an error related to API Key is not valid, please verify that you have entered your key correctly. Additionally, ensure that you have a sufficient number of valid tokens available to access the OpenAI server. You can login OpenAI website to confirm your token Usage status.
3. The demo is initially configured with English as the default chatting language. If you wish to use Chinese, please make the following adjustment in the code: audioTranscription->setLanguage(audioTranscription, "zh"). However, please note that we cannot guarantee that ChatGPT's Chinese conversion and responses will achieve highly accurate results. Please feel free to modify and test this translation to suit your needs.

Resources

Follow the blog posts, demos and tutorials for more updates.

1. BLOG: Unleashing the Power of OpenAI and ESP-BOX: A Guide to Fusing ChatGPT with Espressif SOCs
2. BLOG: OpenAI Component | Accelerating the integration of OpenAI APIs in projects
3. Tutorial: ESP Tutorial: Unleashing the Power of ESP32 S3 BOX 3 with OpenAI

Note

Please note that,

1. To proceed with the demo, you need an OpenAI API key, and you must possess valid tokens to access the OpenAI server.
2. To provide the WIFI credentials and the OpenAI secret key, please follow the on display prompts to proceed.
3. Additionally, as a result of OpenAI's restrictions, this particular example cannot be supported within Mainland China.
4. ChatGPT is a large language model that is unable to distinguish between Chinese Jian ti (简体) and Fan ti (繁體) characters. Due to the constraints of LVGL (Light and Versatile Graphics Library), this project currently only supports Jian Ti (simplified Chinese characters).