PHOBOSLAB
Blog Home

Play GTA V In Your Browser – Sort Of

Inspired by a blog post to run your own cloud gaming service which uses a VPN and Steam's In-Home Streaming, I thought I could do this, too, but in the browser.

With the tech I developed for Instant Webcam I had all the major building blocks ready and just needed to glue them together in nice, useable Windows App.

Demo Video for jsmpeg-vnc

The result is jsmpeg-vnc – a tiny Windows application written in C. It captures the screen at 60 frames per second, encodes it into MPEG1 using FFmpeg, sends it over WebSockets to the browser where it is finally decoded again in JavaScript using jsmpeg. All the mouse and keyboard input in the browser is captured as well and sent back to the server again.

This works way better than it should. Latency is minimal and only affected by the network conditions. It's certainly good enough to play most types of games.

Update: I did some latency measurement now. For my system it hovers around 50-70ms (video, photo). Tested with an 800x600 window at ~7% CPU utilization on a Core i5. My desktop monitor is a Dell U2711, which seems to add about 15ms latency itself.

Full source code and binary releases are on github: jsmpeg-vnc

Monday, July 27th 2015
— Dominic Szablewski, @phoboslab

71 Comments:

#1 – Abhishek – Monday, July 27th 2015, 15:45

WOW!!!

#2 – Roman – Monday, July 27th 2015, 15:48

Wicked cool, man!

#3 – Tom – Monday, July 27th 2015, 15:53

This is cool! Is there a possibility to run this on linux?

#4 – Dominic – Monday, July 27th 2015, 15:55

@Tom: the source is mostly straight C code. Only the part that grabs the raw pixel data from a window is Windows specific, so porting it to another OS shouldn't be all to difficult.

#5 – Kaan – Monday, July 27th 2015, 16:38

This is cool!!!

#6 – Elliot – Monday, July 27th 2015, 16:42

Whoa, that's sick!

#7 – artofcode – Monday, July 27th 2015, 16:54

Nice one.
(offtopic) Is that Anna Fisher on the wallpaper?

#8 – multivac – Monday, July 27th 2015, 17:54

"Latency is minimal"... can we get numbers for the geeks among us?

#9 – Sushil – Monday, July 27th 2015, 18:00

Awesome!!

#10 – Liam – Monday, July 27th 2015, 18:48

This is fantastic. If somebody could port this to Linux, I'd be a very happy man!

#11 – shaan kumar – Monday, July 27th 2015, 19:53

thats cool

#12 – DIEGO – Monday, July 27th 2015, 19:58

This game is cool to play. I played it a million of times. so wanna keep playing it.

#13 – Shaun Trennery – Monday, July 27th 2015, 20:37

Incredible!

#14 – Peter Ivanov – Monday, July 27th 2015, 21:28

G-R-E-A-T !!!!!!!!!!!!

this can enable not only games, but apps too

#15 – jojo – Monday, July 27th 2015, 21:51

haha very nice! your video skills need some major ramp up but oh my goodness this is so cool! :)

#16 – Kev Z – Monday, July 27th 2015, 22:11

This is Sick

#17 – Gideon – Tuesday, July 28th 2015, 04:06

Awesome! It seems like you're running it locally so that's probably why the latency is so low

#18 – Manigandan – Tuesday, July 28th 2015, 06:09

Im getting only 19 fps on wifi network. How to fix that issue

#19 – Bob – Tuesday, July 28th 2015, 10:40

The most impressive part is how this guy manages to play a racing game on his iPhone!

#20 – anugrah – Tuesday, July 28th 2015, 11:39

Awesome stuff!!

#21 – nith proxy – Tuesday, July 28th 2015, 16:01

[394281:4204] ERR: Unable to open 'client/index.html i get this error

#22 – vvtunes – Tuesday, July 28th 2015, 17:59

i'm getting a black screen, what am i doing wrong?

jsmpeg-vnc.exe -s 800x600 -p 8181 cursor
firefox > 192.168.0.142:8181/

(unblocked in windows firewall)

#23 – Sohaeb Mohamed – Tuesday, July 28th 2015, 22:39

how can i make this work for me

#24 – Deryck – Wednesday, July 29th 2015, 02:07

#22 it should start out black while the server works to gather up your video. This will last a few seconds (at least it does for me). You should eventually see what you're looking for.

#23 you may need to be more specific. the instructions on github are very simple and straightforward.

#25 – Rino – Wednesday, July 29th 2015, 03:54

Hi,

i've got error at broswer cannot click mouse while gaming, i'm playing warcraft frozen thrones. this is the error :
Uncaught InvalidStateError: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state.sendMouse @ jsmpg-vnc.js:99(anonymous function) @ jsmpg-vnc.js:109

#26 – Micha – Wednesday, July 29th 2015, 05:55

I am really impressed! I think, you are "underpaid" 8-)

i tested "Dungeon Keeper (FX)" on my iPad Air.
- 800x600

Everything was fine!! *cheers*

One Point: Source (game) Sound-streaming possible?

Greets from Germany

M

#27 – yatin – Wednesday, July 29th 2015, 07:39

[69776:2525] ERR : Unable to open 'client/index.html's')

Can anyone help with this..

#28 – pierre – Wednesday, July 29th 2015, 13:08

Hey Micha kannst du mir vielleicht helfen? ich kenne mich damit zwar nicht aus aber habe auch ein ipad air und würde das gerne mal versuchen ;D

#29 – weest – Wednesday, July 29th 2015, 13:17

Can I stream fullscreen games?

#30 – m – Wednesday, July 29th 2015, 18:54

Wow this is going to be popular, im sure of it

#31 – Rodrigo – Wednesday, July 29th 2015, 22:13

ERR : Unable to open 'client/index.html
Help ?


#32 – Dominic – Wednesday, July 29th 2015, 22:29

The 'Unable to open 'client/index.html' error happens when the jsmpeg-vnc.exe can't find "client/" folder and or contents. Most likely your current working directory in the Dos prompt is not the same as the directory where the .exe is in.

Simply navigate to the jsmpeg-vnc directory first (using 'cd <path>'), before executing the .exe.

#33 – Micha – Wednesday, July 29th 2015, 22:38

@ #28 – pierre –

Hi! Klar! Hast du iMessage am iPad angeschaltet?

#34 – Gabriel Bellon – Thursday, July 30th 2015, 02:18

You're a god! Seriously!

#35 – Igor Gouvea – Thursday, July 30th 2015, 04:16

Someone please help me I'm not getting compile who have step by step help me

#36 – Julian Asgrael – Thursday, July 30th 2015, 15:53

no playable on ps vita, but works

#37 – Jack Wilson – Thursday, July 30th 2015, 16:50

Help!!! I don't know C code. Somebody help me how to use this. Thanks!

#38 – Thamaraiselvam – Thursday, July 30th 2015, 17:17

Awesome man

#39 – Toni – Thursday, July 30th 2015, 23:26

Fantastic man! I'll try it now!

#40 – Glen – Friday, July 31st 2015, 10:01

Is this possible on a Mac?

#41 – Miguel – Friday, July 31st 2015, 11:50

Someone can play there?????????

#42 – Ernesto – Saturday, August 1st 2015, 17:00

If you download only binaries you cannot use. You receive HTPP 404. You also need to add client directory from github

#43 – Alejandro serrano – Tuesday, August 4th 2015, 20:49

Please if I could really play tell me cause I want I so bad thanks

#44 – Needy – Wednesday, August 19th 2015, 15:27

i dont know much about computers so could someone help with playing it in browser

#45 – Help :( !! – Thursday, August 20th 2015, 09:17

How do you open the game ? Let's say I wanted to play DotA. What do I type ?

#46 – Chase – Thursday, September 10th 2015, 02:03

One thing to note is that this is merely over a local network. The latency would be much more significant if used remotely.

#47 – Shakal – Wednesday, October 14th 2015, 18:29

how to install this binaries ?

#48 – brendan – Wednesday, December 2nd 2015, 02:37

Is there any way you could explain this that someone who knows nothing about coding would understand how to set it up and play

#49 – Name – Tuesday, January 5th 2016, 15:23

How can i Play?

#50 – Erik – Friday, January 22nd 2016, 12:38

If it is in C mainly, I believe, that it can't be hard to port it everywhere :) I'm gonna give it a shot with OSX and Linux.

#51 – isolee – Saturday, February 6th 2016, 17:03

amazing! someone port to linux pliz !!!

#52 – antfros – Saturday, February 13th 2016, 10:57

i only get 10-15 fps on gigabit wired network, i7, 16GB ram, WIN10
Cpu only work at 20% i dont understand.

I get 5-6sec of latency :(

#53 – Erik Hric – Monday, April 18th 2016, 20:32

whole desktop works for me, even windows folders are working.
Minecraft shows always only first frame and Return to Castle Wolfenstein only grey screen. Mouse works, but no picture :( any suggestions?

#54 – isaac – Thursday, June 23rd 2016, 23:53

Haven't done nothing.....



#55 – GAmEr – Monday, July 11th 2016, 13:44

where can I get JSMPEG exe file??

#56 – Eliexar – Friday, July 15th 2016, 22:46

Hi giv me da download link plz

#57 – AKA the A – Thursday, September 29th 2016, 10:15

Any chance this could be modified so that the client can run headless (accepts an incoming stream without having to touch the PC)?
Or could you at lest tell me the ffmpeg stream settings you are using?
(couldn't find any examples for mpeg1)

For example I can use FFMPEG to stream mpegts on udp to a particular address and port and the client just has VLC open listening to UDP on the given port...this way I can have the client always on and it starts displaying once the stream arrives and the buffer fills...I can tweak the lag down to about 1.5s with mpeg2, but that's still quite a lot...


I'm trying to come up with a way for office people being able to share their desktop on a headless public terminal with a big TV, but can't use TeamViewer or VNC (security policy)...

#58 – kyvänsyylä – Wednesday, November 30th 2016, 12:13

haista vittu

#59 – luiz gonzaga – Wednesday, January 4th 2017, 17:54

gta v

#60 – mohammad – Sunday, January 8th 2017, 15:08

were is the game

#61 – derik – Wednesday, February 1st 2017, 23:47

Soares

#62 – Dickson – Wednesday, March 22nd 2017, 21:51

wow is this really happening.

#63 – jemus bahati – Saturday, April 8th 2017, 09:24

can l play gta 5 in pc i want to use it

#64 – Alooy – Friday, May 26th 2017, 13:33

hahaha very nice

#65 – henrik – Monday, June 19th 2017, 01:06

Why vnc keyword in project name(jsmpeg-vnc)? Does it use VNC protocol? Is it running VNC server implementation?

#66 – daho_destroyer – Sunday, August 13th 2017, 18:59

this is awsome

#67 – rasjojjlm – Monday, August 14th 2017, 03:51

jsmpeg-unc.exe -s 1920x1000 desktop_

#68 – zermok – Tuesday, October 17th 2017, 04:00

any possibility to replace mpg1 by Mp4 or even
put any kind of protocol like rtmp rtmfp?

#69 – Jack – Sunday, December 17th 2017, 19:54

What is the website to play the game?

#70 – Sambhavjain – Sunday, February 11th 2018, 11:18

this game is so awesome

#71 – ayoub – Sunday, March 18th 2018, 12:33

0669932836