From Pelicans to Ephemeral Worlds
从鹈鹕测试到按需生成世界 · 中英对照 + 共学读书笔记

We're starting to leave the territory where you'd test an LLM by e.g. “create an svg of pelican on a bicycle”.

我们正开始离开这样一种阶段：人们会用诸如“创建一张鹈鹕骑自行车的 SVG 图”这样的任务来测试 LLM

As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it.

作为一种把测试推广到更复杂任务的想法，我很好奇：如果把《指环王》的第一段交给 Opus 5，给它 100 万 Token 的预算，约合 10 美元，并要求它用 Three.js 把这段文字渲染出来，它会做出什么

Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story.

Opus 自主工作了大约两小时，写下 5500 行代码，以程序化方式渲染了这个故事

It's kind of janky but fun.

它有些粗糙，但很有趣

But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all.

但令人有些难以置信的是，LLM 必须把各种多边形资产放置并编排在三维空间的 (x, y, z) 坐标中，还要编写代码让这一切动起来，而它最终居然真的做出了一个能运行的东西

I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from “no one would ever do this” to “sure, why not, it's ~free”.

我也喜欢这类例子，因为任何理性的人都不会花费这么多时间去编写一个定制程度如此之高的东西，但 LLM 仿佛有用不完的耐力和耐心，因此这是一个让我们从“没人会去做这种事”走向“当然可以，为什么不呢，反正几乎免费”的例子

There might be a lot more.

这样的可能性或许还有很多

But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc.

但真正令我兴奋的是创建高度定制的世界，你可以设想把玩家直接放进这些世界，例如在这里进入《指环王》的故事，作为一个旁观剧情的 NPC，或扮演其中某个角色等等

Something like an ephemeral GTA of X on demand.

就像根据任意主题 X，按需生成一个临时存在的 GTA 式开放世界

Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them.

最后一个想法是，世界与游戏这个领域暴露了 LLM 的一个弱点：它们无法轻易检查自己的工作，因为它们还不能高效、原生地感知视频，也无法亲自在这些游戏中游玩

Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank.

在这个例子中，Opus 5 不得不非常缓慢、费力地在不同时间点截取画面，而且它出错了几次，制造出不少粗糙和不协调的结果

An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

这说明模型仍然相当缺乏一些基础能力，例如多模态感知和实际游戏操作能力