IMCDb Forum
Send an answer to a topic: Site/Ad Issues
Subject
Bold [b]Text[/b] Italic [i]Italic[/i] Underline [u]Underline[/u] Strike Out [strike]Strike Out[/strike]
Email [email=nobody@nobody.org]Name[/email] Link [url=http://www.website.com]Text[/url] Anchor [anchor]Name[/anchor] Image [img]http://www.website.com/image.jpg[/img]
Align Left [align=left]Text[/align] Centered [align=center]Text[/align] Align Right [align=right]Text[/align] Text Justify [align=justify]Text[/text]
Color [color=#000000]Text[/color] Highlight [highlight=pascal]Text[/highlight] Widgets Smileys :code: [:code] HTML to BBCode converter Word to BBCode converter
Preview Spell Checker

Copy Paste Cut Select All
Clear Insert Date Insert Time Insert Date and Time Insert your IP
List [list=square][item]BlaBla[/item][/list] Numbered List [list=decimal][item]BlaBla[/item][/list]
Quote [quote=name]Text[/quote] Spoiler [spoiler]James is the murderer![/spoiler]
Uppercase [uppercase]Text[/uppercase] Lowercase [lowercase]Text[/lowercase] l33t [l33t]I'm a Nerd[/l33t] Sub Script [sub]Text[/sub] Super Script [sup]Text[/sup] Size of Text [size=8]Text[/size]

Options
 
 
 
 
antp
Using the country search is not heavy has long as there is a make or model also to first limit the number of vehicles :wink:
And anyway it is not a big problem that users do heavy search, as they usually do one every few seconds/minutes.
The problem is the bots that make a lot of these quickly... and for nothing, since it is just browsing the same vehicles that what they already have via movies :ohwell:
dhill_cb7
I might search a brand "Nissan" "made for GB" or "Ford Mondeo MK1" but never anything elaborate as you posted above.
antp
And I also added a new bot to the list, the 2nd most active of yesterday:
The Meta-ExternalAgent crawler crawls the web for use cases such as training AI models or improving products by indexing content directly.

:kiki:
Even when the bot filter is not active, that list is useful, as some pages are always blocked for bots. This is the case for some of the heaviest pages when they are useless for them (e.g. comment search if they find a link somewhere, or just the "all comments" pages, as they can find the same contents in other ways).
antp
I see in the logs that some nonfilterable bots make searches that are very heavy for the server, e.g. browse all UK vehicles (there are thousands of pages, no human would just browse all that, sometimes several times in the same few seconds)
In a way it is my fault that they find such link: from the statistics page, there are links to show all cars of a given country, but it has no sense for those with more than a few vehicles.
The easiest way to "fix" that is to limit the search options for users that are not logged in. I assume that visitors won't use too much the advanced searches anyway.

Example of queries that I should block:
At the exact same second, 8 person who request the page 4470 of steam engines with UK origin?
That's for sure bots, but why request 8 times the same thing? That looks more like DDOS than just indexing a site.

161.123.110.152 www.imcdb.org - [20/May/2025:02:21:55 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.1875.72"
89.40.81.245 www.imcdb.org - [20/May/2025:02:21:55 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.imcdb.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
93.180.224.189 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
74.110.138.252 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.imcdb.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
72.135.194.213 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
188.241.15.246 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.imcdb.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.1826.33"
204.242.40.254 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.imcdb.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.1826.33"
161.123.90.146 www.imcdb.org - [20/May/2025:02:21:56 +0200] "GET /vehicles.php?class111=111&makeMatch=2&modelInclModel=on&origin=UK&page=4470 HTTP/2.0" 200 33436 "https://www.google.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.4 Safari/605.1.15"
antp
:ohwell: I wonder if it is just due to bots or another problem.

There were many bots this night though.
I see many requests that are most likely from bots, but which do not have a specific user agent, so I can't even filter them easily. Same for the IP, they use different ones.
I'm not even sure I could filter them by limiting the number of requests per user, since they don't look like the same one for the site.
Baube
I noticed that too.. wait like forever to get on it , then a few minutes later it works great , then oops , then works fine , oops some time later... well, we get the idea.. i did not thought of checking the time when oops happened.. :ohwell:
night cub
We seem to have a 40 minute gap between postings. And that was a while after I made that post above.
hachirokutrueno
Site is down for me as well, as of 4:06 pm PST
UKboy205
Me too :sad:
dhill_cb7
Yes down for me as well. EST 6:06 pm
Category:  






Ada
CSS
Cobol
CPP
HTML
Fortran
Java
JavaScript
Pascal
Perl
PHP
Python
SQL
VB
XML
Anon URL
DailyMotion
eBay
Flickr
FLV
Google Video
Metacafe
MP3
SeeqPod
Veoh
Yahoo Video
YouTube
6px
8px
10px
12px
14px
16px
18px
Sign In :: Sign Up :: Lost your login or your password?
KelCommunity.be :: © 2004-2025 Akretio SPRL :: Powered by Kelare