Live intelligence

Latest AI updates

45 updates

Tutorial

AWS CDK construct automates SageMaker AI endpoint auto scaling during updates

The AWS Machine Learning Blog describes a reusable CDK construct that replaces native Application Auto Scaling resources for SageMaker AI endpoints with three CloudFormation custom resources. The construct deregisters scalable targets before an endpoint update and re-registers the target and scaling policy afterward, avoiding update failures and silent loss of auto scaling. Asynchronous completion handlers wait for the endpoint to reach InService and the scalable target to become queryable. A Co

AWS Machine Learning BlogOct 24, 2026Score 71

AWSAmazon SageMaker AIAWS CDK

Tutorial

Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI

AWS's ML Blog publishes a technical walkthrough for reducing voice-agent latency by streaming each completed sentence to a hosted Kokoro-82M TTS model on Amazon SageMaker AI as soon as the agent generates it, rather than waiting for the full response. It covers a bidirectional-streaming SageMaker endpoint, sentence chunking, and two transports for returning audio: base64-in-SSE versus multiplexed binary audio with JSON control frames. The post reports time to first audio improving by more than h

AWS Machine Learning BlogOct 23, 2026Score 78

Amazon SageMaker AIKokoro-82MAmazon Bedrock AgentCore

Tutorial Editor selected

Measuring, not guessing, SageMaker autoscaling targets for TTS endpoints

This AWS Machine Learning Blog post walks through deploying the Kokoro-82M text-to-speech model as a custom container on an Amazon SageMaker AI real-time endpoint, then deriving autoscaling thresholds from two live measurements instead of guesswork. It measures Q_max (the per-instance concurrency ceiling that still meets a 3,000 ms p95 time-to-first-audio SLO, found to be 50 in the sample run) and T_total (end-to-end scale-out time, 260.2 s measured). The tooling then computes scale-out and scal

Why it matters: It gives practitioners a concrete, measurement-driven method and tooling for tuning autoscaling on latency-sensitive TTS inference endpoints, a gap that default LLM- and request/response-oriented guidance does not cover.

AWS Machine Learning BlogOct 23, 2026Score 80

Amazon SageMaker AIKokoro-82Mtext-to-speech

Tutorial

OmniParseBench: an open benchmark for document parsers, rebuilt and audited

This tutorial walks through Datalab's OmniParseBench, an open document-parsing benchmark comprising 16,288 pass/fail tests across 2,937 pages covering scanned forms, handwriting, arXiv math and dense tables. Using the released verdicts for 27 parsers, the author recomputes the leaderboard, inspects how the scorer's unit tests reach a verdict, and tests ranking certainty and per-document-type slices. It reports that the top two parsers are statistically tied, that the leader changes across most s

MarkTechPostOct 23, 2026Score 70

OmniParseBenchDatalabdocument AI benchmarking

Tutorial

Independent CPU Benchmark Pits Cactus Whistle Against Whisper Base for Speech-to-Text

A MarkTechPost coding guide benchmarks Cactus Compute's 16.9 MB Whistle speech-to-text model against Whisper tiny and base on seeded LibriSpeech test-clean and test-other samples, using speaker-cluster bootstraps with Holm correction. The author reports a lead over greedy Whisper base on test-other and no significant gap against beam search, with a speed advantage that narrows on longer clips. The tutorial also documents a 30-second single-pass window where longer audio is silently truncated, pl

MarkTechPostOct 22, 2026Score 70

Cactus ComputeOpenAI Whisperspeech recognition

Tutorial

NVIDIA Nsight Python Tutorial: Profiling GPU Kernels In-Process from Jupyter

MarkTechPost walks through NVIDIA's open-source nsight-python library, which attaches Nsight Compute to a running Python process so developers can annotate code regions and get hardware metrics back as a pandas DataFrame. The tutorial covers run-count and confidence intervals, cache and clock control, a fp32/fp16 matrix-size sweep with a derived TFLOP/s metric, and hardware counters for distinguishing compute-bound from memory-bound kernels. It also demonstrates the upcoming 1.1 features: an exp

MarkTechPostOct 22, 2026Score 68

NVIDIANsight PythonGPU performance profiling

Tutorial Editor selected

AWS details production architecture for its open-source Deep Insight multi-agent system

AWS Solutions Architecture published the first part of a series on Deep Insight, an open-source multi-agent system that turns CSV data and natural-language prompts into validated analysis reports. The post describes eight role-specialized agents (Coordinator, Planner, Supervisor, Coder, Validator, Reporter, Tracker, Plan Reviewer) built with the Strands Agents SDK Graph and Agents-as-Tools patterns, with model selection tiered across Claude Haiku, Opus and Sonnet via Amazon Bedrock. It also outl

Why it matters: It offers concrete, reusable architectural guidance and trade-offs for teams moving multi-agent systems from prototype to production, backed by open-source code.

AWS Machine Learning BlogOct 22, 2026Score 80

AWSmulti-agent systemsagent orchestration

Tutorial

AWS demonstrates hybrid SLM/cloud agent routing with Strands Agents

AWS's Machine Learning Blog walks through the Adaptive Agentic Intelligence Framework (AAIF), built on Strands Agents, which uses a small local orchestrator model (llama3.2:1b via Ollama) to classify requests and route routine tasks to on-device models while escalating complex queries to cloud models on Amazon Bedrock. The post provides working code for local telemetry and vision tools, a Bedrock expert-insights tool, and deployment variants spanning edge, fully cloud, and air-gapped configurati

AWS Machine Learning BlogOct 22, 2026Score 68

AWSStrands AgentsAmazon Bedrock

Tutorial

AWS shows three ways to train Qwen3 on Trainium with torchtitan

AWS published a walkthrough for training Qwen3 on a single trn2.48xlarge using torchtitan-neuron, with TorchNeuron registering Trainium as a native PyTorch device. It covers three configurations: a Transformers v5 modeling backend that reads and writes Hugging Face safetensors, torchtitan's native Qwen3 implementation selected via config.model_spec, and NKI mixture-of-experts kernels attached through torchtitan's override registry. The post also shows how to register a custom NKI rotary-embeddin

AWS Machine Learning BlogOct 22, 2026Score 77

AWS TrainiumtorchtitanQwen3

Tutorial

AWS shows conversational supply-chain agents with Amazon Quick and Bedrock AgentCore

AWS published an end-to-end walkthrough of a conversational supply-chain intelligence pattern in which Amazon Quick serves as the user interface and Amazon Bedrock AgentCore provides the agent runtime, gateways, identity and policy layer. Business systems such as disruption alerts, governance, quoting and invoicing are exposed as MCP servers through AWS Agent Registry, letting an agent plan multi-step tasks and return consolidated impact assessments and mitigation options. The post includes an o

AWS Machine Learning BlogOct 21, 2026Score 69

Amazon Bedrock AgentCoreModel Context Protocolenterprise agent architecture

Tutorial Editor selected

AWS shows SFT-then-GRPO recipe to make GPT-OSS-20B reason in non-English languages

AWS details a fine-tuning workflow on Amazon SageMaker HyperPod that teaches GPT-OSS-20B to reason and answer in French, German, Spanish and Italian instead of defaulting to English. Supervised fine-tuning with LoRA raised reasoning-language accuracy from 22% to 100% on a 50-case evaluation, while adding GRPO on top of the SFT checkpoint lifted answer-language accuracy from 62% to 70%. A controlled Qwen2.5-7B experiment across multilingual reasoning and GSM8K tasks supports the argument that GRP

Why it matters: It offers a concrete, reproducible fine-tuning methodology and decision framework for choosing between SFT and GRPO when adapting open-weight reasoning models to non-English languages.

AWS Machine Learning BlogOct 21, 2026Score 83

Amazon SageMaker HyperPodGPT-OSS-20Breinforcement learning

Tutorial

AWS and Keyfactor detail X.509 PKI identity architecture for AI agents using MCP

An AWS Machine Learning Blog post describes a reference architecture for giving AI agents verifiable, revocable cryptographic identities using Keyfactor EJBCA X.509 PKI on AWS. The design adds signed request envelopes with nonces and timestamps, user-signed delegation assertions, RFC 8693 audience-scoped tokens, and real-time OCSP revocation checks across three trust boundaries: user-to-system, agent-to-agent, and agent-to-MCP-tool-server. It uses Amazon Bedrock with the Strands Agents SDK, Keyc

AWS Machine Learning BlogOct 21, 2026Score 75

AWSKeyfactor EJBCAModel Context Protocol

Tutorial

Eon Centralizes Multi-Cloud Agent Tooling with an MCP Gateway on Amazon Bedrock AgentCore

Eon replaced per-tool credentials and local Model Context Protocol servers with a single authenticated MCP gateway built on Amazon Bedrock AgentCore, giving developers and an autonomous SRE agent unified access to investigation tools across AWS, GCP and Azure. Cedar policies at the gateway enforce least-privilege separation between human and agent identities, and IAM cross-account role assumption removed the need for VPC peering per account. Eon says onboarding dropped from roughly two days to a

AWS Machine Learning BlogOct 20, 2026Score 78

Amazon Bedrock AgentCoreModel Context ProtocolEon

Tutorial

Denodo and Amazon Bedrock AgentCore Semantic Layer for Governed Agentic AI

AWS's ML blog describes a reference architecture that pairs the Denodo Platform with Amazon Bedrock AgentCore to give agents governed access to federated enterprise data. The Denodo MCP Server exposes semantic views to a Strands agent through AgentCore Gateway, while AgentCore Identity performs on-behalf-of token exchange so end-user identity and Denodo row and column security policies are enforced at query time. A customer 360 example federates Amazon Aurora, Amazon Redshift and an on-premises

AWS Machine Learning BlogOct 20, 2026Score 68

Amazon Bedrock AgentCoreDenodo PlatformAI agents

Tutorial

AWS shows semantic layer pattern for agentic AI with Strategy Mosaic and Bedrock AgentCore

AWS published a walkthrough for giving agentic AI agents a governed semantic layer, using Strategy Mosaic over Amazon Redshift and Amazon Aurora with a Strands agent hosted on Amazon Bedrock AgentCore. Mosaic exposes one dimensional model through MCP to agents and through JDBC/ODBC, XMLA, and REST to other clients, with linked models handling cross-source joins without ETL. The post covers metric definitions, named filter objects, hierarchies, and propagation of per-user row-level and object-lev

AWS Machine Learning BlogOct 20, 2026Score 70

Strategy MosaicAmazon Bedrock AgentCoreenterprise semantic layer

Tutorial

AtScale's dimensional semantic layer exposed to Bedrock AgentCore agents via MCP

AWS's latest semantic-layer post in its series swaps AtScale in for Stardog and Denodo, modeling a Customer 360 domain in AtScale's open YAML-based Semantic Modeling Language versioned in Git. A single deployed AtScale model is served simultaneously over SQL, XMLA and MCP, letting a Strands agent hosted on Amazon Bedrock AgentCore and existing BI tools get the same computed answers from shared metric definitions. The article details query planning and pushdown to Amazon Redshift Serverless, auto

AWS Machine Learning BlogOct 20, 2026Score 68

semantic layeragentic AIAWS

Tutorial Editor selected

Push-based KV transfer in vLLM cuts TTFT up to 3x for disaggregated inference

AWS describes a push-based NIXL KV connector for vLLM's prefill-decode disaggregated serving, contributed upstream as an accelerator-agnostic feature. Instead of the decode instance pulling KV blocks after prefill, the decode side registers target blocks in advance and the prefill side issues an NIXL WRITE from a background thread, removing the router round trip and setup work from the TTFT critical path. On a 1P1D deployment with EC2 P5en instances and EFA, the Qwen3-32B benchmark showed up to

Why it matters: It gives concrete benchmark data and implementation guidance for a real vLLM latency optimization that teams running disaggregated inference can adopt.

AWS Machine Learning BlogOct 20, 2026Score 82

vLLMLLM inference servingAWS

Tutorial Editor selected

Bi-directional KV transfer in vLLM cuts TTFT for multi-turn agentic workloads

AWS details a bi-directional KV cache transfer capability in vLLM that lets decode nodes return retained KV blocks to prefill nodes on later conversation turns, avoiding repeated prefill computation in P-D disaggregated serving. It uses an extended NIXL KV connector and a KV-cache-aware router that tracks block metadata across turns, with configurable block retention TTL and a recompute threshold. On a 1P1D Amazon EC2 P5en deployment with EFA, AWS reports TTFT reductions of up to 1.6x for Qwen3-

Why it matters: Gives LLM serving teams a concrete open-source vLLM optimization, with configuration guidance and measured latency data, for reducing time to first token in long multi-turn agent sessions.

AWS Machine Learning BlogOct 20, 2026Score 84

vLLMKV cache transferLLM inference serving