My Projects
Posted: Sun Jun 07, 2015 12:09 am
I feel your pain there. I moved my home external web-server out into the cloud some 6-7 years ago. I was just thinking earlier this year that perhaps I could move it back in house, then some time ago I noticed that the Internet seemed to 'slow down'. Last week I braved Telstra support and found that the Busselton exchange is congested, with no plans to fix/upgrade it, customers/me are currently being shaped down to 6Mb This is back to the old ADSL days
I do love some of the comments they wrote up about your predicament - very witty. Talking of spiders and scrapers, a number of years ago I spent a weekend playing around with my website logs, looking at Drupal page load times, what the robots where looking/crawling around etc. As you say, on small sites, most of the traffic that we pay for are just them - visiting over and over again. I was pondering
about writing some sort of automated firewall system, that would only allow a crawler in once, after a new page was posted, then they would be blocked/firewalled off till next time.Come on NBN with a high upload speed!
