๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ
  • ๐Ÿ‘ฉ๐Ÿปโ€๐Ÿ’ป ๐ŸŒฎ ๐Ÿ’ฌ

๐Ÿ‘ฉ๐Ÿป‍๐Ÿ’ป/network7

[Celery] ๊ฐ„๋‹จํ•˜๊ฒŒ ์„ฑ๋Šฅ ์ฒดํฌ ์‹คํ–‰ ํ›„ CPU ์‚ฌ์šฉ๋Ÿ‰ ๋ฐ Celery Worker ์ƒํƒœ๋ฅผ ๋ชจ๋‹ˆํ„ฐ๋งํ•˜๋ ค ํ•œ๋‹ค. โœ” Celery Worker ํ”„๋กœ์„ธ์Šค ํ™•์ธps aux | grep "celery worker" โœ” CPU & ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰ ์ฒดํฌhtop โœ” Redis์— ๋Œ€๊ธฐ ์ค‘์ธ ์ž‘์—… ํ™•์ธredis-cli llen celery → ๋งŒ์•ฝ ๋Œ€๊ธฐ ์ค‘์ธ ์ž‘์—… ๊ฐœ์ˆ˜ ํ™•์ธํ•˜๊ณ  ์ ์ ˆํ•œ concurrency ์กฐ์ •→ ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰๋„ ๊ณ ๋ คํ•ด์„œ concurrency ์กฐ์ • 2025. 2. 19.
[Celery] ์ ์ ˆํ•œ concurrency ๊ฐ’ ์„ค์ •ํ•˜๊ธฐ ๊ฐœ์š”Celery์—์„œ concurrency ๊ฐ’์€ ํ•œ ๋ฒˆ์— ๋ช‡๊ฐœ์˜ ์ž‘์—…์„ ๋™์‹œ์— ์‹คํ–‰ํ• ์ง€ ๊ฒฐ์ •ํ•˜๋Š” ์ค‘์š”ํ•œ ์„ค์ •์ด๋‹ค.์ ์ ˆํ•œ concurrency ๊ฐ’์€ ์–ด๋–ป๊ฒŒ ์„ค์ •ํ• ๊นŒ .. (ํ•˜๋“œ์›จ์–ด, ์›Œํฌ๋กœ๋“œ, ์ž‘์—… ํŠน์„ฑ์— ๋”ฐ๋ผ ๋‹ค๋ฅด๋‹ค!)ํ™•์ธconcurrencyCelery๋Š” ๊ธฐ๋ณธ์ ์œผ๋กœ Prefork (multiprocessing) ๋ชจ๋“œ๋ฅผ ์‚ฌ์šฉconcurrency๋Š” ๊ฐ worker๊ฐ€ ๋™์‹œ์— ์ฒ˜๋ฆฌํ•  ์ˆ˜ ์žˆ๋Š” ์ž‘์—… ๊ฐœ์ˆ˜๋ฅผ ์˜๋ฏธํ•œ๋‹ค.worker๋Š” ์‹ค์ œ๋กœ Task๋ฅผ ์‹คํ–‰ํ•˜๋Š” ํ”„๋กœ์„ธ์Šค(์ž‘์—… ๋‹จ์œ„)๋ฅผ ์˜๋ฏธํ•œ๋‹ค.1๏ธโƒฃ CPU ์ฝ”์–ด ์ˆ˜Celery๊ฐ€ CPU ์—ฐ์‚ฐ์„ ๋งŽ์ด ์‚ฌ์šฉํ•˜๋Š” ์ž‘์—…(์˜ˆ: ๋จธ์‹ ๋Ÿฌ๋‹, ์ด๋ฏธ์ง€ ์ฒ˜๋ฆฌ ๋“ฑ)์„ ์‹คํ–‰ํ•œ๋‹ค๋ฉด, CPU ์ฝ”์–ด ์ˆ˜ ๊ธฐ์ค€์œผ๋กœ ์„ค์ • CPU ์ฝ”์–ด ๊ฐœ์ˆ˜์ ์ ˆํ•œ concurrency ๊ฐ’2 ์ฝ”์–ด2~44 ์ฝ”์–ด4~88 ์ฝ”์–ด8~.. 2025. 2. 19.
[Proxy] ๋ฆฌ๋ฒ„์Šค ํ”„๋ก์‹œ Reverse Proxy ์ •๋ฆฌ ๊ฐœ์š” ํŠน์ • port์˜ ์„œ๋น„์Šค์—๋งŒ HTTPS 443 SSL ์ ์šฉํ•˜๊ธฐ๊ฐœ์š”์šด์˜์ค‘์ธ ์„œ๋น„์Šค ์ค‘์—์„œ 8003 ํฌํŠธ์—๋งŒ ssl ์„ ์ ์šฉํ•˜๋ ค๊ณ  ํ•œ๋‹ค.์ง„ํ–‰ ์š”์•ฝ๋„๋ฉ”์ธ ์ด๋ฆ„์ด ์—†์ด IP๋งŒ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ OpenSSL๋กœ ์ž์ฒด ์„œ๋ช… ์ธ์ฆ์„œ๋ฅผ ์ƒ์„ฑํ•ด์•ผ ํ•œ๋‹ค.NGINX๋ฅผ HTTPS ์ฒ˜๋ฆฌ ๋‹ด๋‹น์ž๋กœ ๋‘๊ณ ,bonory.tistory.com8003 ํฌํŠธ์—๋งŒ https ์ ์šฉํ•˜๋ฉด์„œ ๋ฆฌ๋ฒ„์Šค ํ”„๋ก์‹œ๋ฅผ ์‚ฌ์šฉํ–ˆ๋‹ค. ์ด ๋ถ€๋ถ„์— ๋Œ€ํ•ด์„œ ์ •๋ฆฌํ•ด๋ณด๋ ค ํ•œ๋‹ค.ํ™•์ธProxy ์„ค์ • ์ฝ”๋“œ:location / { proxy_pass http://127.0.0.1:8003; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'u.. 2025. 1. 24.
ํŠน์ • port์˜ ์„œ๋น„์Šค์—๋งŒ HTTPS 443 SSL ์ ์šฉํ•˜๊ธฐ ๊ฐœ์š”์šด์˜์ค‘์ธ ์„œ๋น„์Šค ์ค‘์—์„œ 8003 ํฌํŠธ์—๋งŒ ssl ์„ ์ ์šฉํ•˜๋ ค๊ณ  ํ•œ๋‹ค.์ง„ํ–‰ ์š”์•ฝ๋„๋ฉ”์ธ ์ด๋ฆ„์ด ์—†์ด IP๋งŒ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ OpenSSL๋กœ ์ž์ฒด ์„œ๋ช… ์ธ์ฆ์„œ๋ฅผ ์ƒ์„ฑํ•ด์•ผ ํ•œ๋‹ค.NGINX๋ฅผ HTTPS ์ฒ˜๋ฆฌ ๋‹ด๋‹น์ž๋กœ ๋‘๊ณ , ํด๋ผ์ด์–ธํŠธ ์š”์ฒญ์„ ๋ฐฑ์—”๋“œ ์„œ๋น„์Šค๋กœ ํ”„๋ก์‹œํ•ด์ฃผ๋Š” ๋ฐฉ์‹์œผ๋กœ ์„ค์ •openssl ์ธ์ฆ์„œ ์‚ฌ์šฉ ์‹œ “์ฃผ์˜ ์š”ํ•จ” ๊ฒฝ๊ณ  ํ™•์ธ → ๋ฌด๋ฃŒ ๊ณต์ธ ์ธ์ฆ์„œ ๋ฐœ๊ธ‰ ์ง„ํ–‰์ธ์ฆ์„œ ์ž๋™ ๊ฐฑ์‹  ์„ค์ •์ง„ํ–‰1. openssl๋กœ ์ธ์ฆ์„œ ๋ฐœ๊ธ‰sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \ -keyout /etc/ssl/private/selfsigned.key \ -out /etc/ssl/certs/selfsigned.crt-x509: ์ž์ฒด ์„œ๋ช… ์ธ์ฆ์„œ๋ฅผ ์ƒ์„ฑ-d.. 2025. 1. 22.
RPC vs gRPC RPC (Remote Procedure Call)ํ”„๋กœ์„ธ์Šค ๊ฐ„ ํ†ต์‹ (IPC)์„ ์œ„ํ•œ ํ”„๋กœํ† ์ฝœ๋กœ์ปฌ ํ™˜๊ฒฝ์—์„œ ํ•จ์ˆ˜ ํ˜ธ์ถœ์„ ํ•˜๋“ฏ์ด ๋„คํŠธ์›Œํฌ๋ฅผ ํ†ตํ•ด ๋‹ค๋ฅธ ์ปดํ“จํ„ฐ๋‚˜ ์„œ๋ฒ„์˜ Procedure(ํ•จ์ˆ˜)๋ฅผ ํ˜ธ์ถœํ•˜๋Š” ๊ฐœ๋…ํ•ต์‹ฌ ๊ฐœ๋…: ๋„คํŠธ์›Œํฌ๋ฅผ ํ†ตํ•ด ์›๊ฒฉ ์„œ๋ฒ„์˜ ํ•จ์ˆ˜๋‚˜ ๋ฉ”์„œ๋“œ๋ฅผ ํ˜ธ์ถœํ•˜์—ฌ ๊ฒฐ๊ณผ๋ฅผ ๋ฐ›๋Š”๋‹ค.์ž‘๋™ ๋ฐฉ์‹:ํด๋ผ์ด์–ธํŠธ๋Š” ํ˜ธ์ถœํ•˜๋ ค๋Š” ํ•จ์ˆ˜์˜ ์ธํ„ฐํŽ˜์ด์Šค๋ฅผ ์•Œ์ง€๋งŒ, ํ•จ์ˆ˜๊ฐ€ ์‹ค์ œ๋กœ ์‹คํ–‰๋˜๋Š” ์„œ๋ฒ„์˜ ์œ„์น˜๋Š” ์ถ”์ƒํ™”๋˜์–ด ์žˆ๋‹ค.์š”์ฒญ์€ ๋„คํŠธ์›Œํฌ๋ฅผ ํ†ตํ•ด ์ „๋‹ฌ๋˜๊ณ , ๊ฒฐ๊ณผ๋Š” ๋ฐ˜ํ™˜๋œ๋‹ค.๊ตฌํ˜„ ๋ฐฉ์‹:XML-RPCJSON-RPCSOAP ๋“ฑ …์žฅ์ :๋กœ์ปฌ ํ˜ธ์ถœ์ฒ˜๋Ÿผ ๋ณด์ด๋„๋ก ์ถ”์ƒํ™”ํ•˜์—ฌ ๊ฐœ๋ฐœ์ž๊ฐ€ ์‰ฝ๊ฒŒ ์‚ฌ์šฉ ๊ฐ€๋Šฅ๋‹จ์ :๋„คํŠธ์›Œํฌ ์„ฑ๋Šฅ์— ์˜์กด๋ฐ์ดํ„ฐ ์ง๋ ฌํ™”/์—ญ์ง๋ ฌํ™”์— ์ถ”๊ฐ€ ๋น„์šฉ ๋ฐœ์ƒ์ƒํ˜ธ ์šด์šฉ์„ฑ ์ œํ•œ(๋‹ค๋ฅธ ์–ธ์–ด๋‚˜ ์‹œ์Šคํ…œ ๊ฐ„ ์—ฐ๊ฒฐ์ด ๋ณต์žกํ•  ์ˆ˜ ์žˆ๋‹ค)gRPCRPC์˜ ํ˜„๋Œ€์ .. 2025. 1. 14.
์ธํ„ฐ๋„ท ์—†๋Š” 2๊ฐœ์˜ PC ํ†ต์‹ ํ•˜๋Š” ๋ฐฉ๋ฒ• ๊ฐœ์š”๋„คํŠธ์›Œํฌ๊ฐ€ ์•ˆ๋˜๋Š” ํ™˜๊ฒฝ์—์„œ PC 2๊ฐœ๊ฐ€ ํ†ต์‹ ํ•ด์•ผ ํ•˜๋Š” ์ƒํ™ฉ์—์„œ ์–ด๋– ํ•œ ๋ฐฉ๋ฒ•์„ ์‚ฌ์šฉํ•  ์ง€์— ๋Œ€ํ•œ search1๋ฒˆ PC : stt, llm์ด ์ง„ํ–‰๋˜๋ฉฐ ๋กœ๋ด‡์— ์ „๋‹ฌํ•  ๋ช…๋ น์„ return2๋ฒˆ PC : ๋กœ๋ด‡ ์‹œ๋ฎฌ๋ ˆ์ดํ„ฐ. 1๋ฒˆ PC๋กœ ๋ถ€ํ„ฐ ๋ช…๋ น์–ด๋ฅผ ์ „๋‹ฌ๋ฐ›์•„ ์‹œ๋ฎฌ๋ ˆ์ดํ„ฐ๋กœ ๊ตฌํ˜„ํ•œ๋‹ค.Socket ํ†ต์‹  (TCP/UDP)์†Œ์ผ“ ํ”„๋กœ๊ทธ๋ž˜๋ฐ์„ ์ด์šฉํ•˜๋ฉด 1๋ฒˆ PC์—์„œ ๋ฐ˜ํ™˜๋ฐ›์€ ๊ฐ’์„ 2๋ฒˆ PC๋กœ ์ง์ ‘ ๋ณด๋‚ผ ์ˆ˜ ์žˆ๋‹ค. ๋กœ์ผ“ ๋„คํŠธ์›Œํฌ์—์„œ ip์™€ port๋กœ ํ†ต์‹ ์ด ๊ฐ€๋Šฅํ•˜๋‹ค.๋น ๋ฅด๊ณ , ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ „์†ก์— ์ ํ•ฉํ•˜๋‹ค.๋ฐฉ๋ฒ•์€ ๊ฐ„๋‹จํ•˜๋‹ค.2๋ฒˆ PC์—์„œ ์†Œ์ผ“ ์„œ๋ฒ„๋ฅผ ์‹คํ–‰ํ•˜๊ณ  ํŠน์ • ํฌํŠธ๋ฅผ ์—ด์–ด๋‘”๋‹ค.1๋ฒˆ PC์—์„œ ํด๋ผ์ด์–ธํŠธ ์†Œ์ผ“์„ ํ†ตํ•ด 2๋ฒˆ PC์˜ ์„œ๋ฒ„๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ „์†กํ•œ๋‹ค.python์—์„œ socket ๋ชจ๋“ˆ ์‚ฌ์šฉํ•˜๋ฉด ๋จํŒŒ์ผ ๊ณต์œ  ๋ฐฉ์‹ (SMP/NFS).. 2024. 11. 28.
[gRPC] gRPC๊ณผ websocket ๋น„๊ต ์ •๋ฆฌ ๊ธฐ์กด์— ๊ฐœ๋ฐœํ–ˆ๋˜ ์„œ๋น„์Šค์—์„œ ํด๋ผ์ด์–ธํŠธ์™€ ์„œ๋ฒ„ ํ†ต์‹ ์„ websocket ์œผ๋กœ ๊ตฌํ˜„ํ–ˆ๋‹ค.๋‹ค ๋๋‚ด๊ณ ๋‚˜์„œ ํด๋ผ์ด์–ธํŠธ, ์„œ๋ฒ„ ํ†ต์‹ ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์ข€ ๋” ์„œ์น˜ํ•ด๋ณด๊ณ  ์‹ถ์—ˆ๋‹ค๊ทธ๋ ‡๊ฒŒ ์•Œ๊ฒŒ๋œ gRPC. ์ •๋ฆฌํ•ด๋ณด์ž โœ… gRPC์™€ WebSocketgRPC์™€ WebSocket์€ ๋‘˜ ๋‹ค ๋„คํŠธ์›Œํฌ ํ†ต์‹  ํ”„๋กœํ† ์ฝœ๋กœ, ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ „์†ก๊ณผ ํด๋ผ์ด์–ธํŠธ-์„œ๋ฒ„ ๊ฐ„ ์ƒํ˜ธ์ž‘์šฉ์„ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•จgRPCGoogle์ด ๊ฐœ๋ฐœํ•œ RPC(Remote Procedure Call) ํ”„๋ ˆ์ž„์›ŒํฌHTTP/2 ๊ธฐ๋ฐ˜์œผ๋กœ ๋™์ž‘ํ•˜๋ฉฐ, ํ”„๋กœํ† ์ฝœ ๋ฒ„ํผ(Protocol Buffers, Protobuf)๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐ์ดํ„ฐ ์ง๋ ฌํ™”๋ฅผ ์ˆ˜ํ–‰ํด๋ผ์ด์–ธํŠธ๋Š” ์„œ๋ฒ„์˜ ๋ฉ”์„œ๋“œ๋ฅผ ํ˜ธ์ถœํ•˜๋“ฏ์ด ์›๊ฒฉ์œผ๋กœ ๊ธฐ๋Šฅ์„ ์‹คํ–‰WebSocketHTML5 ํ‘œ์ค€์œผ๋กœ, TCP ์†Œ์ผ“ ์—ฐ๊ฒฐ์„ ํ†ตํ•ด ์–‘๋ฐฉํ–ฅ ํ†ต์‹ ์„ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•จํ”„๋กœ.. 2024. 11. 25.