Search
List of Categories
Who has read this topic?
Add Reply
Printable version
Activate notificationGeneral » Site/Ad Issues
Reminder of the previous message
Me too 
I noticed that too.. wait like forever to get on it , then a few minutes later it works great , then oops , then works fine , oops some time later... well, we get the idea.. i did not thought of checking the time when oops happened.. 
I wonder if it is just due to bots or another problem.There were many bots this night though.
I see many requests that are most likely from bots, but which do not have a specific user agent, so I can't even filter them easily. Same for the IP, they use different ones.
I'm not even sure I could filter them by limiting the number of requests per user, since they don't look like the same one for the site.
And I also added a new bot to the list, the 2nd most active of yesterday:
Even when the bot filter is not active, that list is useful, as some pages are always blocked for bots. This is the case for some of the heaviest pages when they are useless for them (e.g. comment search if they find a link somewhere, or just the "all comments" pages, as they can find the same contents in other ways).
The Meta-ExternalAgent crawler crawls the web for use cases such as training AI models or improving products by indexing content directly.
Even when the bot filter is not active, that list is useful, as some pages are always blocked for bots. This is the case for some of the heaviest pages when they are useless for them (e.g. comment search if they find a link somewhere, or just the "all comments" pages, as they can find the same contents in other ways).
Using the country search is not heavy has long as there is a make or model also to first limit the number of vehicles 
And anyway it is not a big problem that users do heavy search, as they usually do one every few seconds/minutes.
The problem is the bots that make a lot of these quickly... and for nothing, since it is just browsing the same vehicles that what they already have via movies
And anyway it is not a big problem that users do heavy search, as they usually do one every few seconds/minutes.
The problem is the bots that make a lot of these quickly... and for nothing, since it is just browsing the same vehicles that what they already have via movies
Add Reply