← Back to portfolio
VIEON Humanoid Robot — Software Brain
ROBOTICS / AIOngoing
AI software stack for real-time humanoid control
ESP32-CAMMCU
FreeRTOSFramework
16 chServos
<50msLatency
The software brain handles everything from low-level servo PWM control to high-level AI reasoning. The ESP32 runs FreeRTOS with a dual-core task split — Core 0 handles Wi-Fi, WebSocket, and camera streaming while Core 1 manages the PCA9685 servo driver, MPU6050 PID loop, and sensor fusion. A laptop-based Python server provides Gemini API access, OpenCV vision, and a Next.js dashboard for manual control.
Architecture Layers
- Layer 1 (On-device): ESP32 firmware — servo control, sensor reading, cliff detection, voice command parsing
- Layer 2 (Local network): Python FastAPI server — WebSocket motor commands, OpenCV face/object detection, Gemini API proxy
- Layer 3 (Cloud optional): Gemini API free tier — natural language reasoning, object recognition, task planning
Python Server Stack
- FastAPI — async HTTP + WebSocket server
- OpenCV — face detection, ArUco marker tracking, object recognition
- Gemini API — natural language task planning, vision queries
- uvicorn — ASGI server for production
- Next.js dashboard — live servo sliders, camera feed, gait sequence editor
Gait Control Algorithm
The 4-phase bipedal gait cycle coordinates all 8 leg servos (CH8–CH15) with timed sequences. MPU6050 provides real-time tilt feedback for the PID balance loop that corrects hip roll during each phase.
- Phase 1: Shift weight left — Hip Roll LEFT (CH8) tilts pelvis
- Phase 2: Lift right leg — Hip Pitch RIGHT (CH13) + Knee RIGHT (CH14) raise foot
- Phase 3: Swing forward — Hip Pitch RIGHT (CH13) advances leg
- Phase 4: Place foot, shift right — Hip Roll RIGHT (CH9) rebalances, cycle repeats