You might need to use something that can mimic a

also one more thing. don't exceed the threshold if you do so. using a VPN is not going to always help. you have other fingerprints that can identify you as a bot.

0

11.03.2021

Пользователь 61930 Автор вопроса

Tigran
there uiwebview which opens urls , and then i can ...

but, can you open the uiwebview in the background? a headless chrome is going to run in the background for you.

0

11.03.2021

Tigran

Пользователь 61930
but, can you open the uiwebview in the background?...

it seems the best idea is headless chrome , but is it possible to use it in my ios app?

0

11.03.2021

Пользователь 61930 Автор вопроса

Tigran
it seems the best idea is headless chrome , but is...

no. and it consumes RAM. usually people use this option on small VPSs so that they don't exceed any threshold. and cache the result for downstream use. like maybe expose it via API to the mobile app. crawling is not as easy as it used to be.

0

11.03.2021

Tigran

Пользователь 61930
no. and it consumes RAM. usually people use this o...

they closed their api

0

11.03.2021

Ulven - Old account

Tigran
they closed their api

What API?

0

11.03.2021

Пользователь 61930 Автор вопроса

Tigran
they closed their api

no I meant you have to rent one or more than one VPS and then, use a headless chrome to make the requests. and then cache the result. and then expose that result to your IOS app via your own API.

0

11.03.2021

Пользователь 61930 Автор вопроса

Ulven - Old account
What API?

autotrader.com I presume.

0

11.03.2021

Tigran

Пользователь 61930
no I meant you have to rent one or more than one V...

ohh it is not easy )) , i thought parsing would be an easy task))

0

11.03.2021

Tigran

Ulven - Old account
What API?

auttotrader.com

0

11.03.2021

Пользователь 61930 Автор вопроса

Tigran
ohh it is not easy )) , i thought parsing would be...

no. and also BTW requests is not a good library for crawling and parsing. there are better ways like this one https://www.crummy.com/software/BeautifulSoup/bs4/doc/ you can google python bs4 tutorials the internet is full of them.

0

11.03.2021

Tigran

Пользователь 61930
no. and also BTW requests is not a good library fo...

thank you very much for giving the idea of headless chrom , i found an ios headles browser and now everything works

0

11.03.2021