Macnica and neoAI begin collaboration to build and implement generative AI infrastructure using on-premise "local LLM"

Macnica (Head office: Kohoku-ku, Yokohama; Representative Director and President: Kazumasa Hara; hereinafter referred to as Macnica) and neoAI, Inc. (Head office: Chiyoda-ku, Tokyo; Board Director: Chiba Shunsuke; hereinafter referred to as neoAI) announced today that they will begin collaborating to provide generative AI solutions that make large-scale language models (LLMs) available on-premise. Combining Macnica 's expertise in building AI environments with neoAI's cutting-edge AI technology, the two companies will accelerate the digital transformation (DX) of Japanese companies and promote the practical application of generative AI.

■ Background to the demand for local LLM without using the cloud
The rapid development of generative AI, including ChatGPT, has led to the development of LLM environments in companies. The scope of LLM applications is expanding beyond general use cases such as translation, summarization, and email creation, to include utilizing knowledge accumulated within the company in business operations through Retrieval-Augmented Generation (RAG) technology, which improves the accuracy of information retrieval.
Currently, the mainstream approach is to build an LLM environment on the cloud provided by a public cloud vendor, but when handling highly confidential data such as design/manufacturing process data or customer information, there are concerns about information leakage and security when using the cloud. Against this background, attention is being paid to "local LLM," which can be implemented on-premise without using the cloud. Local LLM not only has security advantages, but also allows specialization for specific company-specific operations and tasks, and in the long term, it also has cost advantages.

■Challenges in establishing local LLMs
Building a local LLM requires a lot of resources, both technical and human, and is a barrier for many companies, starting with procuring GPU resources, building an AI infrastructure that requires specialized knowledge of LLM inference/learning/RAG, and then developing and maintaining applications and managing the GPU infrastructure after construction. In addition, to effectively utilize generative AI, it is necessary to address issues such as improving accuracy according to use cases, improving user-friendly UI/UX, and spreading it throughout the organization.

■Generative AI solutions provided by Macnica and neoAI
Macnica, with its expertise in building AI environments, and neoAI, with its extensive track record and know-how in using generative AI, have formed a comprehensive support system to help companies use generative AI, from construction to implementation and accompanying support.

Macnica will provide optimal hardware and software that accelerates the development and deployment of generative AI, and will build an environment based on LLM.
neoAI provides an on-premise generative AI environment including RAG, making full use of cutting-edge local LLMs such as the latest Llama 3.1, Gemini 2, and even neoAI's proprietary high-performance Japanese LLM. By introducing neoAI's all-in-one generative AI package for corporations, "neoAI Chat ^*," it is possible to deploy high-quality AI solutions while reducing costs and shortening implementation time compared to scratch development. Furthermore, by utilizing neoAI's learning know-how, we provide a wide range of support, including the construction of LLMs that have been additionally trained with company-specific data.

Even after implementation, Macnica and neoAI will provide ongoing, detailed support, such as improving accuracy and expanding functions in response to the unique challenges faced by each company. Through this solution, they will accelerate the digital transformation of Japanese companies and strongly promote the practical application of generative AI and the improvement of corporate competitiveness.

<neoAI product screen example>

Regarding this announcement, Shunsuke Chiba, Board Director of neoAI Inc., said the following:
"At neoAI, we have been promoting the use of generative AI in areas where high security is required, such as financial institutions. As we have progressed with our generative AI project, we have received many requests to build LLM environments that take into consideration not only the cloud but also on-premise. We are excited to be able to provide an on-premise LLM environment in collaboration with Macnica, from building the GPU environment to applications."

James Hsieh of Super Micro Corporation (Headquarters: Shibuya-ku, Tokyo, Country Manager and VP Sales: James Hsieh), the Japanese subsidiary of Super Micro Computer, Inc. (Headquarters: California, USA, CEO: Charles Liang), which supports the partnership between the two companies with AI-oriented GPU servers, which are important for the construction and development of generative AI infrastructure, said the following.
"We are very pleased to be able to cooperate on the hardware side in this AI infrastructure construction collaboration between neoAI and our domestic distributor Macnica. Supermicro's GPU servers are always at the forefront of the global AI market, and we will support this initiative to commercialize generative AI in Japan by providing cutting-edge GPU-equipped AI servers."

Keisuke Ito, Macnica CLAVIS Company Inc., said the following:
"We are very pleased to begin providing generative AI infrastructure construction and implementation support using cutting-edge local LLM through our partnership with neoAI. Macnica has contributed to the introduction of AI development platforms for our customers. We are confident that by starting this collaboration with neoAI, we will be able to provide support for the implementation and utilization of high-quality generative AI."

*: neoAI Chat is an all-in-one generative AI package provided by neoAI, and is used by over 50 companies. By simply uploading a document, you can create a generative AI assistant specialized for each company's business. In addition to high-performance RAG functions, it also has the management functions necessary for introducing generative AI in-house.

[For product inquiries, please contact here]
Macnica CLAVIS Company Company Product Manager
Tel: 045-470-9821

*Company names and product names mentioned in the text are trademarks or registered trademarks of Macnica, neoAI, Inc. and the respective companies.
*Information published in news releases (including product prices, specifications, etc.) is current as of the date of announcement. Please note that it may be subject to change without prior notice.