์ „์ฒด๋ณด๊ธฐ 494

Pandas Data Frame vs Spark DataFrame

DataFrame ์€ ํ–‰๊ณผ ์—ด์ด ์žˆ๋Š” ๋ฐ์ดํ„ฐ ํ…Œ์ด๋ธ”์„ ๋‚˜ํƒ€๋‚ด๋ฉฐ, DataFrame ๊ฐœ๋…์€ ์–ด๋–ค ํ”„๋กœ๊ทธ๋ž˜๋ฐ ์–ธ์–ด์—์„œ๋„ ๋ณ€ํ•˜์ง€ ์•Š์ง€๋งŒ Spark ์™€ Pandas ์˜ DataFrame ์€ ์ƒ๋‹นํžˆ ๋‹ค๋ฅด๋‹ค. ์ด ๊ธ€์—์„œ๋Š” Spark DataFrame๊ณผ Pandas DataFra,e์˜ ์ฐจ์ด์ ์„ ์•Œ์•„๋ณด๋ ค๊ณ  ํ•ฉ๋‹ˆ๋‹ค. Pandas DataFrame Panda๋Š” NumPy ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•˜๋Š” ์˜คํ”ˆ ์†Œ์Šค Python ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์ž…๋‹ˆ๋‹ค. ๋‹ค์–‘ํ•œ ๋ฐ์ดํ„ฐ ๊ตฌ์กฐ์™€ ์—ฐ์‚ฐ์„ ์‚ฌ์šฉํ•˜์—ฌ ์ˆ˜์น˜ ๋ฐ์ดํ„ฐ์™€ ์‹œ๊ณ„์—ด์„ ์กฐ์ž‘ํ•  ์ˆ˜ ์žˆ๋Š” Python ํŒจํ‚ค์ง€์ž…๋‹ˆ๋‹ค. ์ฃผ๋กœ ๋ฐ์ดํ„ฐ ๊ฐ€์ ธ์˜ค๊ธฐ ๋ฐ ๋ถ„์„์„ ์ƒ๋‹นํžˆ ์‰ฝ๊ฒŒ ํ•˜๊ธฐ ์œ„ํ•ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. Panda DataFrame์€ ๋ ˆ์ด๋ธ”์ด ์ง€์ •๋œ ์ถ•(ํ–‰ ๋ฐ ์—ด)์„ ๊ฐ€์ง„ ์ž ์žฌ์ ์œผ๋กœ ์ด์งˆ์ ์ธ 2์ฐจ์› ํฌ๊ธฐ ๊ฐ€๋ณ€ ํ‘œ ํ˜•์‹ ..

IT Talks/BigData 2023.03.16

๋™ํƒ„-์ธ์ฒœ๊ณตํ•ญ ๋ฒ„์Šค ์‹œ๊ฐ„ํ‘œ (8837๋ฒˆ)

๋…ธ์„  ๋™ํƒ„์—ญ - ๋ผ๋งˆ๋‹คํ˜ธํ…” - ์‹ ๋ผ์Šคํ…Œ์ด - ๋‹ค์€๋งˆ์„ - ๋Šฅ๋™ ์ฃผ๊ณต - ๋ณ‘์ ์ค‘์‹ฌ์ƒ๊ฐ€ - ์ธ์ฒœ๊ณตํ•ญ (T1) - ์ธ์ฒœ๊ณตํ•ญ (T2) ์‹œ๊ฐ„ํ‘œ (๋™ํƒ„์—ญ ์ถœ๋ฐœ) (2023-03-09 ๊ธฐ์ค€, ํ‰์ผ/์ฃผ๋ง ๋™์ผ) ์‹œ ๋ถ„ 4 00, 30 5 15, 45 6 30 7 00, 50 8 30 9 30 10 10 11 10, 50 12 50 13 30 14 40 15 30 16 30 17 10 18 15 19 05 (์ถœ์ฒ˜ - ๋ฒ„์Šคํƒ€๊ณ ) 2023-03-09 ๋ถ€ํ„ฐ ์ฆ์ฐจ (10ํšŒ -> 20ํšŒ) ๋™ํƒ„~์ธ์ฒœ๊ณตํ•ญํ–‰ 8837 ์‹œ์™ธ๋ฒ„์Šค ์ฆ์ฐจ์šดํ–‰ ๊ณตํ•ญ๊นŒ์ง€ ๊ฐ€๋Š” ๊ธธ ๊ฑฑ์ •์—†๋‹ค์š”! ๐Ÿ˜‰ ๐ŸšŒ ๋™ํƒ„~์ธ์ฒœ๊ณตํ•ญ ์ฆ์ฐจ์šดํ–‰ ์˜ค๋Š” 3์›” 9์ผ๋ถ€ํ„ฐ ๋™ํƒ„๊ณผ ์ธ์ฒœ๊ณตํ•ญ์„ ์˜ค... blog.naver.com ์ฐธ๊ณ  ๋งํฌ - ๋„ค์ด๋ฒ„ ์ง€๋„ ๋ฒ„์Šค ์ •๋ณด : https://naver...

Travel 2023.03.09

๊ฐค๋Ÿญ์‹œ S23U ๋กœ ์ฐ์€ ๋‹ฌ๋‹˜

์˜ค๋žœ๋งŒ์— ํฐ์„ ๋ฐ”๊พธ๊ณ  ๋‚˜์„œ Elon Musk ์ฒ˜๋Ÿผ Wow๋ฅผ ์˜ฌ๋ฆฌ๊ณ  ์‹ถ์—ˆ์œผ๋‚˜ ๋‹ฌ์ด ์ž˜ ์•ˆ๋ณด์—ฌ์„œ ๋ชป์ฐ๋‹ค๊ฐ€ ์—„์ฒญ ์–‡์€ ์ดˆ์Šน๋‹ฌ์ด ๋ณด์—ฌ์„œ ์ฐ์–ด๋ณด๊ณ  ๊ทธ ๋’ค๋กœ ๋‹ฌ์ด ๋ณด์ผ๋•Œ ๋งˆ๋‹ค ์ดฌ์˜ํ•ด๋ดค๋‹ค. 10๋ฐฐ ๊นŒ์ง€๋งŒ ๊ด‘ํ•™ ์คŒ์ด๊ณ  ๊ทธ ์ดํ›„๋Š” ๋””์ง€ํ„ธ ์คŒ์ด๋ผ๊ณ  ๋“ค์—ˆ๋Š”๋ฐ, 30๋ฐฐ ๊นŒ์ง€๋„ ๋ณผ๋งŒํ•œ ๊ฒƒ ๊ฐ™๊ณ , 100๋ฐฐ๋Š” ์ข€ ํ๋ฆฟํ•ด ๋ณด์ธ๋‹ค. ์ฐ๊ณ  ๋‚˜์„œ ์ €์žฅํ•˜๋ฉด์„œ ์•Œ์•„์„œ ๋ณด์ •๊นŒ์ง€ ํ•ด์„œ ์•Œ์•„๋ณผ์ˆ˜๋Š” ์žˆ๋Š” ์ •๋„์ธ๋“ฏ ํ•˜๋‹ค.

TALKs 2023.03.06

์•ฑ์—์„œ ์ ‘์†ํ•˜๋Š” URL ํ™•์ธํ•˜๊ธฐ

ChatGPT ์—๊ฒŒ ๋ฌผ์–ด๋ดค๋‹ค. Q) How can I capture the API URL of smartphone application ? A) To capture the API URL of a smartphone application, you will need to use a tool called a network sniffer or packet capture tool. This type of tool intercepts and captures the network traffic between your smartphone and the application's server, allowing you to inspect the requests and responses that are sent and rec..

IT Talks/AI 2023.03.03

์„œ๋ฒ„์˜ ๊ณต์ธ IP ํ™•์ธํ•˜๊ธฐ

์ผ๋ฐ˜์ ์ธ ๋ฆฌ๋ˆ…์Šค ์„œ๋ฒ„๋Š” ์ด๋”๋„ท๋ณ„๋กœ IP๋ฅผ ํ• ๋‹นํ•˜์—ฌ ifconfig ๋ช…๋ น์–ด๋กœ ํ• ๋‹น๋œ IP ๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ์œผ๋‚˜ AWS (Amazon Web Service)๋“ฑ์˜ ํด๋ผ์šฐ๋“œ ์žฅ๋น„์—์„œ๋Š” Private IP ๋งŒ ํ™•์ธ์ด ๊ฐ€๋Šฅํ•˜๊ณ  Public IP ๋Š” Amazon Console ๋“ฑ์„ ํ†ตํ•ด์„œ ํ™•์ธ์ด ๊ฐ€๋Šฅํ•˜๋‹ค. ๋˜๋Š” NAT ์•„๋ž˜์— ์œ„์น˜ํ•œ ์„œ๋ฒ„๋“ค์˜ ๊ฒฝ์šฐ๋„ ์‹ค์ œ ํ• ๋‹น๋œ IP๋Š” ์‚ฌ์„ค IP๋งŒ ํ‘œ์‹œ๋˜๊ณ , ์™ธ๋ถ€ ์ ‘์†์‹œ ์™ธ๋ถ€ ์„œ๋ฒ„์—์„œ ์ธ์‹ํ•˜๋Š” IP ๋Š” NAT ์˜ IP ๊ฐ€ ๋œ๋‹ค. ์ด๋Ÿฐ ๊ฒฝ์šฐ ์„œ๋ฒ„์—์„œ ๊ฐ„๋‹จํ•˜๊ฒŒ ์™ธ๋ถ€๋กœ ์ ‘์†ํ•˜๋Š” ๋ณธ์ธ IP๋ฅผ ํ™•์ธํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ์•„๋ž˜์™€ ๊ฐ™๋‹ค. 1. ํ˜„์žฌ ์‚ฌ์šฉ์ค‘์ธ ์„œ๋ฒ„์˜ ๊ณต์ธIP ์กฐํšŒ (Outbound) $ curl ip.ojj.kr Your IP is 175.123.191.3 2. ํŠน์ • IP์˜ ์ •๋ณด ์กฐํšŒ $ cu..

IT Talks/Linux 2023.02.22

S23U ๋กœ ์ฐ์€ ๊ณ ์–‘์ด - ํ˜ธ๋‘

์šฐ๋ฆฌ์ง‘ ๊ณ ์–‘์ด๋“ค ์ค‘์— ๊ท€์š”๋ฏธ๋ฅผ ๋‹ด๋‹นํ•˜๋Š” ํ˜ธ๋‘. ์ฑ…์ƒ์œ„์— ์˜ฌ๋ผ๊ฐ€์„œ ๋…ธ๋ ค๋ณด๋Š” ์‚ฌ์ง„์„ ์ฐ์–ด์„œ S23์— ์žˆ๋Š” ์ถ”์ถœํ•˜๊ธฐ(๋ˆ„๋ผ ?) ๊ธฐ๋Šฅ์œผ๋กœ ๋ƒฅ์ด๋งŒ ์˜์˜ฅ~ ์ƒ๊ฐ๋ณด๋‹ค ์ถ”์ถœ์ด ์ž˜๋˜๋Š”๋“ฏ.

Animal 2023.02.21

[๋ฆฌ๋ˆ…์Šค] date ๋ช…๋ น์œผ๋กœ timestamp ๋ณ€ํ™˜ํ•˜๊ธฐ

1. timestamp ๊ฐ’์œผ๋กœ ์›ํ•˜๋Š” ๋‚ ์งœ ํ˜•ํƒœ๋กœ ๋ณ€ํ™˜ํ•˜๊ธฐ $ date -d '1970-01-01 1676360315 sec' 2023. 02. 14. (ํ™”) 07:38:35 KST $ date -d '1970-01-01 1676360315 sec' +'%Y-%m-%d %H:%M:%S' 2023-02-14 07:38:35 * ๊ฒฐ๊ณผ๋Š” ์‹œ์Šคํ…œ Timezone์— ๋งž์ถฐ์„œ ํ‘œ์‹œ๋จ 2. ๋‚ ์งœ ํ˜•์‹์œผ๋กœ timestamp ๊ฐ’ ๊ตฌํ•˜๊ธฐ (1๋ฒˆ๊ณผ ๋ฐ˜๋Œ€) $ date -d '2022-02-14' +%s 1644764400

IT Talks 2023.02.14

ChatGPT ๊ฐ€ ์ถ”์ฒœํ•˜๋Š” ๊ฐœ๋ฐœ์ž ์ปค๋ฆฌ์–ด

Q. Please recommend a career as a software developer. A. A career as a software developer can be a very rewarding and fulfilling choice. Here are some reasons why: 1. High demand: Software development is a rapidly growing field with a high demand for skilled professionals. This means that there are many job opportunities available and the demand for software developers is expected to continue to..

IT Talks/AI 2023.02.13

ChatGPT ์—๊ฒŒ ๊ฐœ๋ฐœํˆด์„ ์ถ”์ฒœํ•ด ๋‹ฌ๋ผ๊ณ  ํ–ˆ๋‹ค.

(์˜๋ฌธ) Q. What is the best IDE for developers ? A. The best IDE (Integrated Development Environment) for a developer largely depends on the individual's specific needs and requirements. Some popular IDEs include: 1. Visual Studio Code: It's a free, open-source code editor developed by Microsoft, that supports multiple programming languages and platforms. 2. PyCharm: A dedicated Python IDE develope..

TALKs 2023.02.13
728x90