Hi For a scrapper program (at large scale) which scrapes

Question

Hi For a scrapper program (at large scale) which scrapes

over 12,000 online shopping websites with more than 10M pages per day, is this a good idea to use python ?
Or is this a good idea to use go ?

i would appreciate it if you could give me your opinion 🙏

0

20.11.2020

21 ответов

11 просмотров

Daddy Автор вопроса

Roman Sharkov
Python may not be the best idea if you care about ...

I think the biggest obstacle for us is the I/O Speed. Some of my friends said there is no difference between the two. But the development speed in Python is faster. So they suggested me to use Python. But I'm still not sure

0

20.11.2020

Roman Sharkov

Daddy
I think the biggest obstacle for us is the I/O Spe...

that might be true, however, I don’t have enough details to give you a useful answer

0

20.11.2020

ㅤ

Roman Sharkov
that might be true, however, I don’t have enough d...

you live in heaven man

0

20.11.2020

Daddy Автор вопроса

Roman Sharkov
that might be true, however, I don’t have enough d...

https://juliensalinas.com/en/crawling-large-volumes-of-web-pages/ I found enough reasons to use go instead of python in this arrticle thank you 🙏

0

20.11.2020

ㅤ

Daddy
https://juliensalinas.com/en/crawling-large-volume...

also, Python bites. Go doesn't

0

20.11.2020

Liλn 🌻

ㅤ
also, Python bites. Go doesn't

😂 good one

0

20.11.2020

ㅤ

Liλn 🌻
😂 good one

😁

0

20.11.2020

Roman Sharkov

ㅤ
also, Python bites. Go doesn't

but when it bites - it bites entire legs off

0

20.11.2020

ㅤ

Roman Sharkov
but when it bites - it bites entire legs off

I'd rather lose my legs than die.

0

20.11.2020

ㅤ

Roman Sharkov
but when it bites - it bites entire legs off

ⓘ 𝗢𝗳𝗳𝗶𝗰𝗶𝗮𝗹 𝘀𝗼𝘂𝗿𝗰𝗲𝘀 𝘀𝘁𝗮𝘁𝗲𝗱 𝘁𝗵𝗮𝘁 𝗶𝘀 𝗳𝗮𝗹𝘀𝗲 𝗮𝗻𝗱 𝗺𝗶𝘀𝗹𝗲𝗮𝗱𝗶𝗻𝗴

0

20.11.2020

Roman Sharkov

ㅤ
ⓘ 𝗢𝗳𝗳𝗶𝗰𝗶𝗮𝗹 𝘀𝗼𝘂𝗿𝗰𝗲𝘀 𝘀𝘁𝗮𝘁𝗲𝗱 𝘁𝗵𝗮𝘁 𝗶𝘀 𝗳𝗮𝗹𝘀𝗲 𝗮𝗻𝗱 𝗺𝗶𝘀𝗹𝗲...

what’s false and misleading?)

0

20.11.2020

ㅤ

Roman Sharkov
what’s false and misleading?)

fake news, your message got labeled lol

0

20.11.2020

Roman Sharkov

ㅤ
fake news, your message got labeled lol

where? 😂

0

20.11.2020

ㅤ

Roman Sharkov
where? 😂

joking, recently Trump's twits have been getting labeled with such notifications. It's become a pun

0

20.11.2020

Roman Sharkov

ㅤ
joking, recently Trump's twits have been getting l...

ah 😅

0

20.11.2020

ㅤ

Daddy
https://juliensalinas.com/en/crawling-large-volume...

Thanks for this, it had a reference to a book called Designing Data Intensive Applications. I'll read it.

0

20.11.2020

Daddy Автор вопроса

ㅤ
Thanks for this, it had a reference to a book call...

I only read the first 4 lines of this article :lol: Now I want to read the rest. But I think this article was a good reason to use Go instead of Python. + Also I should read this book, the book name looks fantastic

0

20.11.2020

ㅤ

Daddy
I only read the first 4 lines of this article :lol...

You'll have fun using go for your purpose. Reading the book is a must I think

0

20.11.2020

Mark X

ㅤ
You'll have fun using go for your purpose. Reading...

Data intensive applications?

0

20.11.2020