Explore tools, prompts, skills, news and resources related to model inference.
Latest articles, prompts, use cases and editorial content about model inference
AWS published a walkthrough for configuring the ChatGPT desktop app and Codex to use OpenAI models through Amazon Bedrock in AWS GovCloud (US-West). The setup routes inference through the Bedrock Mantle endpoint in us-go...
Worldwide Express (part of ShipStation Global) built a multi-agent system on Amazon EC2 with Amazon Bedrock inference that works directly in its SpeedShip transportation management system to resolve LTL freight invoice e...
The InferD paper introduces a low-interaction honeypot framework for studying threats targeting AI inference infrastructure, including model-serving stacks, inference APIs, vector databases and agent gateways. It emulate...
NVIDIA announced that its cloud partners across Europe, the Nordics, Africa and Central Asia are expanding regional compute capacity for training and inference of agentic and physical AI workloads. The company cited Nsca...
Postman and AWS detail the architecture behind Postman's Agent Mode, an AI agent running on Amazon Bedrock that works across API testing, documentation, discovery, and implementation. At scale, the team found contex...
NVIDIA announced a new 64GB unified memory configuration of its DGX Spark personal AI supercomputer, available October 23 starting at $4,999 from Acer, ASUS, Dell, Gigabyte, HP and MSI. The system keeps the GB10 Grace Bl...
The AWS Machine Learning Blog details serving quantized GGUF models with the AWS Deep Learning Container for llama.cpp on Amazon SageMaker AI, exposing OpenAI-compatible chat completions through InvokeEndpoint. It then e...
AWS describes how to invoke OpenAI GPT-6 Sol, Luna, and Astra through Amazon Bedrock global cross-Region inference profiles from Singapore, Jakarta, Malaysia, Thailand, and Taipei. The post shows configuration through th...
AWS described three kernel-level optimizations for fused Mixture-of-Experts inference on NVIDIA Hopper: a Swap-AB transposition to improve WGMMA utilization, routing FP8 tensor-quantized MoE to the Triton backend, and el...
"Full Attention is Being Forgotten"With the widespread adoption of AI Agents driving the demand for long-sequence processing, the Attention mechanism in traditional GPT architectures is increasingly viewed as a...
Microsoft's messaging for the new Surface Laptop Ultra is notably restrained: a 15-inch device powered by Nvidia's RTX Spark on an Arm architecture, optimized for Windows 11. The head of Microsoft Surface has bol...
Alibaba, Google, Huawei and ByteDance Enter the Fray: Who Will Win the Era of the "Silicon-Based Employee"?On May 13, Alibaba Group released its financial results for the fourth quarter and full fiscal year 202...
Discover tools, products and platforms related to model inference
