-
内容大纲
在音频、视频以及复杂数据分析等领域,AI技术的巨大潜力尚未被充分挖掘。事实上,由于缺乏指导和实际应用案例,很多当今的专业人士在将AI创新成果应用于这些多元领域时仍面临挑战。
这本全面指南填补了这一空白,专为中高级机器学习工程师、数据科学家和研究人员量身打造。作者Nicole Koenigstein带领读者深入探索Transformer模型的多样化应用,不仅深化理论理解,更着重强调了面向实际应用的可操作策略。本书将提供关于Transformer的“大统一理论”——这些基础洞见将确保你始终立于技术前沿,无论最先进的模型如何演变。
通过本书,你将学会将Transformer应用于:
图像、视频、音乐生成等非文本领域。
推理模型、代码智能体以及多智能体架构。
训练阶段与推理阶段的优化策略。
生产部署、运行时工程以及硬件效率优化。 -
作者介绍
尼科尔·柯尼希施泰因是智能体系统领域的AI研究员与实践者,活跃于科研、咨询、教学以及系统落地等多个领域,致力于构建可靠、可用于生产环境的AI系统。她著有AI Agents:The Definitive Guide(O'Reilly 2026年出版)等多部著作,曾担任欧盟委员会AI大挑战(European Commission AI Grand Challenge)项目的外部评审专家。 -
目录
Preface
1.From First Principles to State-of-the-Art Transformers
Transformer Basics
Tokenizer: Text Representation in the Transformer
Token and Positional Embeddings
Attention Mechanism
Encoder and Decoder Parts
Enhancements in Transformer Design:
Longer Context and Attention Variations
Longer Context Windows with Better Performance
Attention Mechanism Variations
Conclusion
2.Transformers for Time Series
Understanding the Intricacies of Time Series Data
Autocorrelation and Partial Autocorrelation
Cointegration
Cross-Correlation
Stationarity
Trend and Seasonality
Preparing a Dataset
Time Series Modeling in Various Application Domains
Tokenizing Time Series Data
Chronos: Learning the Language of Time Series
Fine-Tuning Chronos
PatchTST: A Time Series Is Worth 64 Words
Fine-Tuning PatchTST on Historical IBM Stock Prices
TimesFM: A Decoder-Only Time Series Foundation Model
Fine-Tuning TimesFM on Hourly Energy Consumption Data
AnomalyBERT for Self-Supervised Anomaly Detection
Conclusion
3.Transformers for Vision Tasks
Overview of Different Vision Tasks
Embeddings and Tokenization for Vision Models
Key Strategies for Improving the Robustness and Effectiveness of Vision Tasks
Swin Transformer V2
Image classification with Swin Transformer V2
Segment Anything
Fine-Tuning SAM on a Custom Dataset
Segment Anything in Images and Videos
Segment Videos and Images with Concept Prompts
Conclusion
4.Transformers for Image Generation
Introduction to Generative Image Models
Diffusion Models: What's That Noise About?
Classifier-Free Guidance in Diffusion Models
Scalable Diffusion Models with Transformers
Generating Images with the DiT
PIXART-α
Generating Images with PixArt-Σ
Diffusion Vision Transformers for Image Generation
Interpretable Features with Diffusion Transformers
Conclusion
5.Transformers for Video Generation
Hidden Effectiveness of Latent Diffusion
LTX-Video: Video in Realtime
Latte: Structured Detail, Poured into Every Video Frame
Tora: From Trajectory to Storyline, One Frame at a Time
Conclusion
6.From Sound to Token and Back: Transformers in the Audio Domain
From Waveforms to Spectrograms:
Understanding the Structure of Audio Data
Audio as a Waveform
Sampling Rate and the Nyquist Theorem
Amplitude, Bit Depth, and Quantization
The Frequency Domain and Fourier Transform
Spectrograms and the Short-Time Fourier Transform
The Mel Spectrogram and Perceptual Scaling
Phase, Reconstruction, and Vocoders
Audio Modeling in Various Application Domains
Transformer Architectures for Audio:
From Perception to Foundational Intelligence
The Rise of Speech Transformers: The Impact of Whisper
Audio Foundation Models: Unifying Understanding, Generation, and Conversation
Qwen2-Audio
Transcribing a Meeting with Kimi-Audio
Segment Anything in Audio
Beyond Text and Speech: Transformers as Music Composers
Conclusion
7.Reinforcement Learning Transformers
Getting Started with Reinforcement Learning
Foundational Concepts in Reinforcement Learning
Online and Offline Reinforcement Learning
Model-Based and Model-Free Approaches
On-Policy Versus Off-Policy Reinforcement Learning
Temporal Difference Learning
World Models in Reinforcement Learning
Transformers in Reinforcement Learning
Decision Transformer
Going Live: Online Decision Transformer
A Brave New World: Stochastic Transformer-Based World Model
TWISTER: Transformer-Based World Models with Contrastive Predictive Coding
Conclusion
8.Embracing the Era of Experience: Transformers for Planning, Reasoning, and Coding
From Human Data to Lived Experience
Learning to Reason: From Pretraining to Reinforcement Learning
DeepSeek-R1: Reinforcing Reasoning Capabilities
Qwen3: Unified Reasoning with Dynamic Control
Qwen3-Coder: Agentic Reasoning for Open-Ended Coding
Kimi K2: Open Agentic Intelligence at Scale
Muon: Scaling Optimization for the Agentic Era
Inference with Kimi K2
Scaling Reasoning at Test-Time: Smarter, Not Just Bigger
Adaptive Branching Monte Carlo Tree Search (AB-MCTS)
The RethinkMCTS Framework for Code Generation
The S* Framework for Code Generation
Conclusion
9.From Scripts to Thinking: AI Agents for Complex Tasks
Autonomy: What's Possible at the Moment?
Designing Agent Workflows
Multi-Agent Architectures
Agentic Communication: The Right Context Is All You Need
Beyond Context: How to Help Agents Remember
Agent Memory Types
Going Global and Lifelong
The Human Factor: Steering Agent Actions
Common Patterns for Human-in-the-Loop
Solving GitHub Issues with Coding Agents
Conclusion
10.Smarter, Better, Faster, Stronger: Optimizing LLMs and AI Agents
Training-Time Intelligence: Reinforcement Learning for Agents
Beyond Hand-Crafted Rewards: How RULER Works
Training in Practice: ART in a Market Scenario
Reason Smarter, Not Harder: Adaptive Compute Allocation
The Delta Incentive: Enforcing Efficiency
Open Innovation: Community-Driven RL Frameworks
The Checkpoint Engine: Systems-Level Optimization for LLM Policy Updates
Conclusion
11.Deploying Transformer Models
Choosing Between Open and Closed Source
Understanding the Architecture You're Deploying
Deploying Decoder-Only Models
Runtime Engineering for Decoder-Only Models
Security Considerations for Decoder-Only Deployments
Building Applications with Coding Models
Evaluating LLM Deployments in Production
Cost Efficiency and Hardware Comparison
Quantization
Test-Time Low-Rank Adaptation in Vision-Language Models
Conclusion
12.Where to Go Next: From Models to Intelligent Systems
Combining Capabilities: SAM 3 Agent
The Science of Scaling Agentic Systems
Conclusion
Index
同类热销排行榜
- C语言与程序设计教程(高等学校计算机类十二五规划教材)16
- 电机与拖动基础(教育部高等学校自动化专业教学指导分委员会规划工程应用型自动化专业系列教材)13.48
- 传感器与检测技术(第2版高职高专电子信息类系列教材)13.6
- ASP.NET项目开发实战(高职高专计算机项目任务驱动模式教材)15.2
- Access数据库实用教程(第2版十二五职业教育国家规划教材)14.72
- 信号与系统(第3版下普通高等教育九五国家级重点教材)15.08
- 电气控制与PLC(普通高等教育十二五电气信息类规划教材)17.2
- 数字电子技术基础(第2版)17.36
- VB程序设计及应用(第3版十二五职业教育国家规划教材)14.32
- Java Web从入门到精通(附光盘)/软件开发视频大讲堂27.92
推荐书目
-

孩子你慢慢来/人生三书 华人世界率性犀利的一枝笔,龙应台独家授权《孩子你慢慢来》20周年经典新版。她的《...
-

时间简史(插图版) 相对论、黑洞、弯曲空间……这些词给我们的感觉是艰深、晦涩、难以理解而且与我们的...
-

本质(精) 改革开放40年,恰如一部四部曲的年代大戏。技术突变、产品迭代、产业升级、资本对接...
[
