Good evening, colleagues! I need help with own technical question as: What

technologies can be used for parser, susceptible to any variation in website design (DOM)?

Today this question will seem strange to someone, but believe me, tomorrow it will be irreplaceable thing, if you support to beat it🔥

I open to discuss.

Thank you for the attention.

You can use liberties like bs4 & scrapy for the same

They're asking for a solution so that they don't have to rewrite their scraper when the website changes.

Does it work even if the whole design of the website will change, am I correctly understand your point here?

If the whole site changes, then no.

I need something that will work even the whole website will change the design. Selenium is working only with specific design version.

Idk if that's even possible

Nope!

It should be possible, at least, with NLP

Tine to use or train a LLM to do it for you

Стикер

Exaxtly

+ might be illegal for some spesific websites

Why llm? There surely must be another type of model more applicable to the use case?

I bring them new clients, and twice as many, I don’t see a problem with unhappy requests to the site, it sounds funny compared to how many new clients would come to them.

bots*

A) read what they said again B) https://t.me/thedevs_chat/636436

exactly

If for such sites it is such a big money for sent requests, why do they position themselves as large sites with serious turnover, judging by what is written about them? Or are they so broke that they can't afford new clients? Do you believe this yourself?

Can you guarantee it?

Of course, I worked with clients from Dubai, I know what I'm saying

Not possible at least that it implemts AI

You can check easily, what are they providing on international level, as I see they are online retailer specializing in consumer electronics, gadgets, and men's fashion. The company provides products in various other categories like home & garden, bags, baby & kids, health & beauty and more from brands like Apepal, Zanflare, Utorch, Apple and more, and so on? So they can't find money on requests in exchange for new clients, right?

But those sites don't know it. And if you can prove what you say to them, it would be better to get to an agreement with the sites to get the data directly from them without scraping.

Imagine if multiple scrappers make multiple requests to the site, it can add up quickly. And as I said before, those site owners can't know for sure if what you say is true.

I got a few things to say about this question, too… for one, a link to quora where you asked the same thing doesn't count as research, and I'd have to check SO guidelines, but I think they don't like when people use chatgpt or similar as the main/only source.

Research is to show the amount of tools, which could be used. Does it makes sense to show each website of these tools if I provide the list of it?

I prove when they received money from the first client I received % from it, so I think this is a clear proof.

But thank you a lot for imputs