System configuration

We used the IMU built into the micro:bit. Sensor data is sent from the micro:bit to the NVIDIA DGX Spark™ via BLE UART, and then converted to the MQTT protocol and published by the micro:bit MQTT bridge. The data is subscribed to within the Node-RED flow to detect changes in activity. A NemoClaw message server is set up to notify NemoClaw in real time of detected change events.

Real-time event notifications

Event notification method adopted

There is a command called ` nemoclaw sandbox exec` that allows you to execute commands within the Box.

Using this feature, you can run `openclaw agent --agent main --message user message` within the Box to instantly deliver the message to the AI agent in NemoClaw.

This command will be executed from Node-RED running within a Docker container, so we will turn it into an API endpoint.

- Because the AI agent cannot determine the recipient of a reply to messages delivered using this method, you need to specify the channel ID of the Discord channel to which the reply should be sent using SKILL.

 

IMU data transmission

The following is the code to send IMU data via BLE UART.

Node-RED flow

Node-RED The following features were implemented in the flow:

MQTT subscriber
• Dashboard (chart display)
• Activity monitoring (change detection)

Dashboard

You can monitor IMU sensor values in real time on the dashboard.

Activity Monitor

The code to monitor activity and detect changes was implemented using Node-RED functions. From the perspective of ensuring real-time performance, it is better to leave the handling of detected change events to NemoClaw 's AI agent rather than placing this functionality within NemoClaw itself. The following is the function that monitors activity.

// ===== function ノード: 動作切り替わり検知 ===== // 入力 msg.payload: { accl_x, accl_y, accl_z, gyro_x, gyro_y, gyro_z } (1サンプル) // 出力: 切り替わりが起きた時だけ msg を送出 const WIN = env.get("AD_WINDOW"); // 窓サイズ(サンプル) const HOP = env.get("AD_HOP"); // 何サンプルごとに判定するか const MIN_HOLD = env.get("AD_MIN_HOLD"); // 状態確定に必要な連続回数 // しきい値(キャリブレーション値で上書き推奨) const TH_HIGH = flow.get("thHigh") ?? 200.0; const TH_LOW = flow.get("thLow") ?? 10.0; // --- リングバッファ取得 --- let buf = flow.get("imuBuffer") || []; const p = msg.payload; const accMag = Math.sqrt(p.accel.x * p.accel.x + p.accel.y * p.accel.y + p.accel.z * p.accel.z); buf.push({accMag}); if (buf.length > WIN) buf.shift(); // 古いものを捨てる flow.set("imuBuffer", buf); // --- HOPごと & 窓が満ちた時だけ評価 --- let cnt = (flow.get("hopCount") || 0) + 1; flow.set("hopCount", cnt); if (buf.length < WIN || cnt < HOP) return null; flow.set("hopCount", 0); // --- 特徴量(加速度の標準偏差 + 角速度平均) --- const accArr = buf.map(b => b.accMag); const mean = accArr.reduce((a, b) => a + b, 0) / accArr.length; const variance = accArr.reduce((a, b) => a + (b - mean) ** 2, 0) / accArr.length; const accStd = Math.sqrt(variance); const score = accStd; flow.set("score", score); // --- ヒステリシス + 最小保持で状態判定 --- let st = flow.get("state") ?? 0; // 0=STILL, 1=ACTIVE let cand = flow.get("cand") ?? 0; let hold = flow.get("hold") ?? 0; let target = (st === 0) ? (score > TH_HIGH ? 1 : 0) : (score < TH_LOW ? 0 : 1); if (target === cand) hold++; else { cand = target; hold = 1; } let event = null; if (cand !== st && hold >= MIN_HOLD) { const label = (cand === 1) ? "STILL->ACTIVE" : "ACTIVE->STILL"; st = cand; event = { payload: { label }, topic: "motion/event" }; } flow.set("state", st); flow.set("cand", cand); flow.set("hold", hold); return event;

Notification from AI Agent

By specifying the Discord channel ID in SKILL and instructing the notification method, users will be notified automatically when changes in activity are detected, without requiring any user inquiries.

Inquiry

In Episode 4, we explained how to use an acceleration sensor (IMU) to detect changes in activity and notify an AI agent in real time.
In Episode 5, we will review the content of the series so far and discuss future challenges in utilizing IoT sensor data with NemoClaw.