2025-01-18# AI-enabled Cloud gamingAI-enabled cloud gaming seems like one of the
hardest applications to do on cloud rather than locally. However I expect it'll
get done in 10 years.If you're a game developer you might want to work on this.
Why? - Latency limits of human body - Video output - Most people can't disting
uish individual frames in video above 90 frames per second (~10 ms / frame) -
Audio ouput - Some audio engineers on reddit find 10 ms latency when playing mus
ic digitally to be noticeable but acceptable. - Keyboard + mouse input - Human
motor reaction times are generally estimated above 100 ms. Upper bound on nerve
conduction velocity is around 120 m/s, covering 1 metre of neurons from hand to
brain requires >10 ms. Anticipating inputs and reacting to them can lower respo
nse time (often happens in games). - End to end - Many cloud gamers have repor
ted on reddit that <10 ms latency is where FPS and other action-heavy games feel
as fast as playing them offline. - Internet bandwidth limits - streaming 24x7
video requires lot more bandwidth than text/image/audio - 1 gbps fiber connec
tion (with no upload/download cap) is becoming increasingly popular in US, which
is more than sufficient to stream UHD 90 fps video. - Streaming 3D content di
rectly is not possible though. VR headsets-based use cases might (?) still prefe
r streaming 3D content over the rendered 2D output, I haven't studied VR well en
ough. - Latency limits of computers - Input/output device latency - 1 ms laten
cy (1000 Hz) has been achieved on keyboards and mice, and gamers generally feel
increased latency won't be detectable. - Game engine, 3D rendering latency - I
don't know much about this, but seems doable for most games today in under 1 ms
? It depends a lot on the exact application though, there's definitely lots of 3
D apps that can't be built with 10 ms latency constraint. - Network roundtrip
latency - <10 ms has already been achieved on consumer fiber connections in many
US cities, there's no fundamental reason paying customers can't get this in cit
ies across the world. Light travelling from Netherlands to California (9000 km)
one-way takes 33 ms, in practice roundtrip latency is reported around 100 ms (50
ms one-way). As long as the closest datacentre is within 1000 km, there's no ph
ysical limitation on achieving <10 ms via fibre connection. - AI inference tim
e - As per Jacob Steinhardt, [forward passes can be significantly parallelised](
https://bounded-regret.ghost.io/how-fast-can-we-perform-a-forward-pass/). 1 fram
e of video can be generated in under <1 ms, assuming you build an ASIC just for
that specific model. - AI inference cost - This is the biggest bottleneck. Dif
fusion models use maybe 0.1-1.0 PFLOP for 100 sampling steps, for one frame. At
90 fps that's 10-100 PFLOP per second of video generation. For 1 second per seco
nd output, you need a GPU cluster with 10-100 PFLOP/s. H200 is 4 PFLOP/s fp8 ren
table at $2/hour. Assuming Epoch AI scaling laws of FLOP/s/dollar doubling every
2.5 years, we should get 16x more FLOP/s/dollar in 10 years, so 100 PFLOP/s ren
table at $2/hour.Effects on IT ecosystem - If this application can be done on cl
oud, then almost any application can be done on cloud - Cybersecurity and user c
ontrol will be the only reasons to do things locally, performance will no longer
be a reason. Financial incentives to build anything in favour of security or us
er control are a lot weaker than the incentives in favour of higher performance.
Big Tech will no longer need to fund open source software for performance-based
reasons, hence open-source software could lag behind. - Client device could al
so change. End state of this vision is 99.9% of people own machines with touchsc
reen (keyboard+monitor) and network card (but no CPU, no disk, no RAM) and it is
not practical to do anything unless you submit the job to a server. (This will
probably a Big Tech server, unless small cloud is able to compete on getting low
latency connections with ISPs who have inherent network effects.). See example
of mobile being more locked down than desktop, but having more users. It is poss
ible to live without a phone, but you lose access to jobs, friendships, etc. and
are at a disadvantage relative to everyone else. - This incentive structure mak
es it technically less challenging for the NSA (or its equivalent in your countr
y) to get 99% surveillance over people's thoughts. As of today they need to back
door lots of devices, routers and cables, and send whatever is useful back to th
eir servers. This might be possible technically but requires more developer time
and coordination/coercion of intermediaries to pull off. - Incentives push in t
he direction of them using this data for political purposes and also leaking the
data itself.