Overview of this series

NVIDIA NemoClaw is a set of open blueprints for running AI agents more securely and continuously. NemoClaw runs AI agents such as OpenClaw in a sandbox Box, and in this context, a blueprint refers to a YAML package that defines the entire Box environment.

Autonomous AI agents do more than just answer questions; they understand situations, plan, and invoke necessary tools to execute tasks. NemoClaw integrates runtime policy control, model routing, skill execution, state management, and observability to handle these AI agents in near-production environments, making it easier to move from prototype to controlled deployment.

In the realm of IoT, it is crucial to connect data collected from sensors—such as temperature, vibration, current, images, and logs—to on-site decision-making and maintenance activities. Using NemoClaw, you can design a configuration where an AI agent refers to sensor data and related documents to explain signs of anomalies, organize potential causes, and suggest items to check next. Furthermore, because data access and external communications can be controlled by policies, it is also suitable for use cases where safety and accountability are required, such as process control and equipment management.

In this series, we will treat NemoClaw as a "safe and secure execution platform for AI agents" and explain, step by step, how to apply it to the collection, analysis, and anomaly detection of IoT sensor data, using a demo application that we have experimentally built.

In the first chapter, we will first clarify the relationship between AI agents and IoT, and then explain the design for connecting sensor data to NemoClaw. Currently, NVIDIA NemoClaw supports four types of AI agents: OpenClaw, Hermes, LangChain Deep Agent Code, and Pi Coding Agent. However, this article assumes the OpenClaw version of NVIDIA NemoClaw.

How AI agents understand IoT data

NemoClaw can process various file formats, including text, images, videos, and PowerPoint presentations, depending on the agent and model used. Of course, the AI model being used must support these formats. However, NemoClaw cannot directly handle environmental data such as temperature, humidity, and atmospheric pressure emitted from IoT sensors, or data such as vibration and pressure that monitor the state of equipment. Therefore, sensor data must be converted into text and made into a format that the AI agent can handle. Also, IoT sensor data is generally delivered using protocols such as MQTT, but NemoClaw does not support protocols specific to the IoT field such as MQTT, so it needs to be converted to a protocol that NemoClaw can use. In the second episode, we will introduce how to handle MQTT data with NemoClaw.

Furthermore, for an AI agent to correctly understand the data, it also needs instructions from the user (human) regarding what the input data is and how it should be interpreted.

Design for connecting sensor data to NemoClaw

IoT applications continuously monitor sensor data, detect changes in the state of the true target (such as manufacturing equipment) from that data, and, when detected, take the optimal action or prompt the user to take action according to predetermined rules. There are use cases where immediate action is required in response to changes in the state of the true target (real time), and use cases where the transition to action can occur periodically.

Use cases that involve taking action at specific intervals (pull type) will be covered in episodes 2 and 3, while real-time use cases (push type) will be covered in episode 4.

Demo environment

The topics covered in this series are actual NemoClaw It has been verified in the environment. NemoClaw It works in a wide range of environments, but the environments used for testing are as follows:

NVIDIA DGX Spark™ Compatible GB10 Onboard computer MSI EdgeXpert 4TB
▹NVIDIA DGX Spark Assuming that compatible machines will function similarly, this series will cover the following topics: DGX Spark This is what I will write.
NVIDIA NemoClaw (OpenClaw version)
vLLM Runs on a server 6-35b-a3b-nvfp4 Model (DGX Spark Local running on top LLM)
・ User notifications are Discord Use the channel

Demo applications introduced in this series

*1: NemoClaw polls for the occurrence of an event. *2: An external program notifies NemoClaw of the occurrence of an event.

Inquiry