[ 一日30分 인생승리의 학습법 ] Build Your Own Remote Desktop Application using Javascript, Python & WebRTC — Part 2

Build Your Own Remote Desktop Application using Javascript, Python & WebRTC — Part 2

What we going to do ?

We going to build a simple web page which will share your screen on the browser and in that we will integrate mouse onclick events which will get the events from the browser and it will be executed on your desktop using “Python”.

Note : Basically Screen Share API will only allow us to share the screen and its audio. It won’t provide the control functionalities like mouse movements or onclick events. So we are going to add the control feature manually.

Requirements

  • Python 3.4 or above
  • HTML & JavaScript ????

Python Libraries

Let us code . . . ????

Step 1 : Create a client web page with Share Screen Share API

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <script src="./app.js"></script>
    <link href="./style.css" rel="stylesheet">
</head>
<body>
    <video id="vid" 
    autoplay 
    onclick="pointer(event)" 
    width="1920" height="1080">
</video>
</body>
</html>

style.css

body {
    display: flex;
    justify-content: center;
}

app.js

/* code to connect with websocket server */
const socket = new WebSocket('ws://localhost:5000')
/* code to get screen display */
navigator.mediaDevices.getDisplayMedia({video: true, audio: true})
.then(stream => {
    // inject stream object as a source for video element //
    document.getElementById('vid').srcObject = stream
    document.getElementById('vid').play
    // inject stream object as a source for video element //
})
.catch(e => console.log(e))
/* The below function send the mouse coordinates to the server */
function pointer(e) {
    let posX = document.getElementById('vid').offsetLeft
    let posY = document.getElementById('vid').offsetTop
    let X = e.pageX - posX
    let Y = e.pageY - posY
    let pointer = X.toString() + ',' + Y.toString()
    console.log(pointer)
    socket.send(pointer)
}

Note : The above javascript code will act as a client and it will send the mouse coordinates (click event) to the server.

Step 2 : Create a websocket server to execute a mouse movement on desktop using Python

implementer.py

import asyncio      # importing asyncio library #
import websockets   # importing websocket library #
import pyautogui    # importing pyautogui library ## Asynchronous function receive the message from client #
async def handler(websocket, path):
    async for message in websocket:
        try:
            pointer = message.split(',')
            pyautogui.moveTo(int(pointer[0]), int(pointer[1]))
            print('X : ', pointer[0])
            print('Y : ', pointer[1])
        except:
            pass# Iinitating the server #
start_server = websockets.serve(handler, "localhost", 5000)# Run's the server continuously #
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Note : In the above python program we are receiving the coordinates as a string instead of getting it as an object and splitting it by passing “comma” as an argument. There is a reason behind sending and receiving the coordinates in “string” format. Because in this example we are integrating the mouse click event functionalities using websockets but in future will establish a remote connection between two different computers using “WebRTC” instead of websockets or socket.io based on that transferring of object using WebRTC may not work. Hence we are transferring the coordinates as a string.

How it works ?

We need to execute python script first and it will wait for the client to connect and then run the web page. Once the web page loaded on the browser it will ask us to select the window to share. After clicking the window it will start to share screen and at the same time the websocket connection will be established.

We need to choose the window
Need to zoom out the browser to 50%. Because we specified the video size as 1920 x 1080.

Note : We need to zoom out the browser to 50%. Because we specified video size as 1920 x 1080 hence in default the web page will be large in size so we need to resize it using browser zoom out option.

After zooming out to 50% you will see your screen on the browser. Now if you click anywhere on the video element the cursor will automatically move to that position on your main desktop.

That’s it… Done.

Finally we added the mouse onclick functionality to Screen Share API. ????

경축! 아무것도 안하여 에스천사게임즈가 새로운 모습으로 재오픈 하였습니다.
어린이용이며, 설치가 필요없는 브라우저 게임입니다.
https://s1004games.com

Important Note :

The Part 1 & Part 2 blogs are mainly posted to get some idea about developing Remote Desktop Application. In upcoming posts we will develope Remote Desktop Application with full functionalities like Mouse Events, Keyboard Events & On-Screen JoyPad Controls (To play PC games on Mobile).

The latest blog links will be updated here soon…

Thank You & Happy Learning ????

[출처] https://ragul-harisankar.medium.com/build-your-own-remote-desktop-application-using-javascript-python-webrtc-c77b32c0249e

 

 

 

 

 

 

본 웹사이트는 광고를 포함하고 있습니다.
광고 클릭에서 발생하는 수익금은 모두 웹사이트 서버의 유지 및 관리, 그리고 기술 콘텐츠 향상을 위해 쓰여집니다.
번호 제목 글쓴이 날짜 조회 수
1220 민사소송에서의 무고죄 [자료출처: 법률구조관리공단] secret 졸리운_곰 2015.05.07 0
1219 [一日30分 인생승리의 학습법] [Git] 다중 리모트를 사용하여 여러 Git 연동하기(Gitea, GitHub) file 졸리운_곰 2025.05.25 7
1218 [一日30分 인생승리의 학습법] 양자컴퓨팅 프로그래밍 file 졸리운_곰 2025.06.03 12
1217 [一日30分 인생승리의 학습법] Qiskit 시작하기 (Getting Started with Qiskit) file 졸리운_곰 2025.06.03 16
1216 Linked List implementation in Visual Basic 졸리운_곰 2020.03.24 18
1215 [一日30分 인생승리의 학습법] [GitHub][terminal] 비밀번호 인증 에러를 토큰으로 해결하고 로그인 하기 file 졸리운_곰 2025.05.24 20
1214 Programmers > #25 winter recruit > #1 [JAVA] 졸리운_곰 2020.03.08 21
1213 [ 一日30分 인생승리의 학습법] */ git 사용자 이름/패스워드 Repository별 저장 졸리운_곰 2023.03.05 21
1212 [Programmers] #8 가운데 글자 가져오기 [Python] 졸리운_곰 2020.02.25 22
1211 [Programmers] #14 다리를 지나는 트럭 [JAVA] 졸리운_곰 2020.02.27 22
1210 Programmes > #23 문자열 내 마음대로 정렬하기 [Python] 졸리운_곰 2020.03.08 22
1209 [ 一日30分 인생승리의 학습법] 소프트웨어 개발하다보면 "connection reset" 등, 소프트웨어 버그 적인 문제가아닌 하드웨어나 네트워크 오류 메시지의 예 file 졸리운_곰 2025.03.01 22
1208 [ 一日30分 인생승리의 학습법] IT 국비교육, 쓰레기 속에서 그나마 덜 쓰레기인 곳 찾는 팁 file 졸리운_곰 2025.03.08 22
1207 [알고리즘 문제풀이]#3 최댓값과 최솟값 [JAVA] 졸리운_곰 2020.02.22 23
1206 [Programmers] #18 문자열 내 p와 y의 개수 [Python] 졸리운_곰 2020.03.06 23
1205 Programmers > #28 winter recruit > #2 [JAVA] 졸리운_곰 2020.03.23 23
1204 Programmers > Sort > #21 H-Index [JAVA] 졸리운_곰 2020.03.07 24
1203 [ 一日30分 인생승리의 학습법] Apache 2.2에서 2.4로 마이그레이션 시 발생하는 에러 및 해결책 정리 졸리운_곰 2022.12.18 24
1202 [ 一日30分 인생승리의 학습법] 프로그래밍 스킴 Scheme 1 다운로드부터 문법 대부분을 314초만에 알려줄게요 졸리운_곰 2023.01.05 24
1201 [ 一日30分 인생승리의 학습법] Streamline Your MLOps Journey with CodeProject.AI Server : CodeProject.AI 서버로 MLOps 여정을 간소화하세요 file 졸리운_곰 2023.11.25 24
대표 김성준 주소 : 경기 용인 분당수지 U타워 등록번호 : 142-07-27414
통신판매업 신고 : 제2012-용인수지-0185호 출판업 신고 : 수지구청 제 123호 개인정보보호최고책임자 : 김성준 sjkim70@stechstar.com
대표전화 : 010-4589-2193 [fax] 02-6280-1294 COPYRIGHT(C) stechstar.com ALL RIGHTS RESERVED