Macnica and Fixstars provide comprehensive support for Tensorrent products, from real-world evaluation to AI model optimization, deployment, and operation. Their joint proof-of-concept (PoC) reduces the token generation time per token for OpenAI's "gpt-oss-120b" by 31.3%.

Macnica (Headquarters: Yokohama City, Kanagawa Prefecture, Representative Director and President: Kazumasa Hara, hereinafter referred to Macnica) ALTIMA Company and Fixstars Corporation (TSE Prime: 3687, Representative Director and President CEO: Satoshi Miki, hereinafter referred to as Fixstars), a leading company in performance engineering technology, will jointly support domestic companies considering the introduction of Tensorrent AI products, from actual evaluation of AI models to performance optimization, construction of AI infrastructure, implementation, and operation.

Furthermore, as an initiative to verify the effectiveness of technical support through collaboration between the two companies, a joint Proof of Concept (PoC) was conducted on Tensorrent's AI workstation "TT-QuietBox" to analyze and optimize the inference performance of OpenAI's open-weighted large-scale language model "gpt-oss-120b". As a result, the expert matrix multiplication of MoE (Mixture of Experts), which was the main bottleneck in the decoding process, was sped up by 3.68 times, and TPOT (Time per Token after Response Start) was reduced by 31.3%.

This collaboration allows companies to test their planned AI models on actual hardware without having to commit to product deployment. Furthermore, it provides an environment where companies can quantitatively determine whether the AI platform is suitable for their specific applications and performance requirements based on the results of the testing and performance optimization.

■ Background of the collaboration
As the use of generative AI in businesses expands, there is a growing need to use AI models in an environment managed by the company itself, from the perspectives of confidentiality, governance, and operational flexibility. On the other hand, for companies to introduce new AI hardware, it is necessary not only to select the product, but also to verify the operation of the AI model to be used, to perform performance verification on actual equipment, to adjust the software, to build the AI infrastructure, and to provide technical support after implementation.

■Support provided by both companies
Therefore, the two companies will combine Macnica 's AI infrastructure construction and product evaluation support with Fixstars' performance engineering expertise to jointly support everything from actual device evaluation and performance optimization to implementation and operation.

<Main support provided by both companies>
• Actual device evaluation of AI models and applications used by customers
• Building an evaluation environment and a model execution environment.
• AI model porting, operational verification, and performance analysis
• Identifying processing bottlenecks
• Improvement of software implementation, including calculation processing and data transfer methods.
• Optimizing the inference performance of AI models
• AI infrastructure construction, implementation, and operational support
• Technical collaboration with Tensorrent

This allows customers to compare and evaluate AI platforms based on their application and performance requirements, and then consider implementing an environment that suits their needs after reviewing the results of performance optimization.

■ Overview and results of the joint PoC
The two companies conducted a joint proof-of-concept (PoC) to improve the inference performance of OpenAI's open-weight model "gpt-oss-120b" on Tensorrent's AI workstation "TT-QuietBox" (equipped with eight Wormhole chips and a total of 96GB of GDDR6 memory).

Macnica provided TT-QuietBox, a remote evaluation environment, an AI model execution environment, and technical support, while Fixstars conducted a detailed analysis of the AI model's processing status. Based on this analysis, they identified areas where processing time was long and improved the computation process and data transfer methods.

As a result, we sped up the expert matrix multiplication of MoE, which was the main bottleneck in the decoding process, by 3.68 times. This reduced the overall TPOT of the model from 144.85 ms to 99.49 ms, a reduction of 31.3%.

In addition, the decoding speed per user improved by 45.7%, from 6.9 tok/s to 10.05 tok/s. Furthermore, the average response time (E2EL) for all requests was reduced by 23.0%, from 25.0 seconds to 19.3 seconds.

Through this Proof of Concept (PoC), both companies have accumulated technical expertise in analyzing and optimizing large-scale AI models on Tensorrent products. Detailed evaluation conditions, analysis methods, optimization content, and measurement results will be presented in a technical report published by Fixstars.

Fixstars Performance Engineering Series 06:
Practical implementation of accelerating gpt-oss-120b inference in Tenstorrent TT-QuietBox

■Value Proposition and Future Developments
This collaboration combines Macnica 's AI infrastructure construction and product evaluation support with Fixstars' performance engineering to not only provide AI hardware, but also run the AI models that customers actually want to use on the hardware and analyze the processing details. Furthermore, based on the analysis results, we will help improve the software implementation and make more effective use of the AI hardware's performance.

Moving forward, both companies will leverage the knowledge gained from this Proof of Concept (PoC) to provide support not only for gpt-oss-120b but also for a variety of AI models and workloads that customers wish to use. Furthermore, by supporting everything from AI platform selection and performance optimization to implementation and operation, they will contribute to further promoting the use of AI in companies.
Macnica 's Tensorrent product and technology information

■Comments from both companies
Representatives from both companies commented as follows regarding this collaboration and the current Proof of Concept (PoC):

Macnica
Hideomi Ozaki, Vice President, ALTIMA Company
As the options for AI models and hardware expand, companies need to identify the combination that best suits their specific applications and performance requirements. Macnica will connect Fixstars' software optimization technology with Tensorrent's open AI platform to support product selection, real-world evaluation, performance improvement, and implementation/operation. Through this collaboration between the three companies, we will contribute to creating an environment where customers can confidently utilize new AI technologies.

Fixstars Corporation
Satoshi Miki, Representative Director and President CEO
Fixstars' expertise in performance engineering lies in techniques that maximize the performance of AI models by adapting them to the characteristics of the hardware. By improving productivity more efficiently using computing resources of the same scale and performance, capital investment and operating costs can be reduced, further enhancing the return on investment for companies' AI.
Through the Company collaboration with Macnica and Tensorrent, we will support our customers in selecting and utilizing their AI infrastructure.