northwest Wayback machine archiver is now being blocked from accessing this forum
Welcome, Guest. Please login or register.
Sunday January 22 2017 21:13:22 AEDT PM
Home Help Login Register
News: forum firewall bad behaviour Cloudflare Bot scout
Welcome to the Australianelectronicgoldprospectingforum founded in July 2010, an add free totally independent forum with over 70 boards and paid for and managed by the Admin.Total forum Topics:8,582 Total forum Posts:40,044 Members: 612 Total page views:7,258,084 Admin and  forum and domain name owner :marjen at optusnet.com.au. Guests can only see a limited number of boards at present and cannot see any links. Guest cannot post and never will be permitted too!Registration of new members must be approved by admin.Anyone known to have any past or present association with Codan/ML or acting on their behalf as a proxy or intermediary  will not have their registration approved. All  original Photos and posts and  original materials displayed on this site are COPYRIGHTED and remain the property of the poster and the australianelectronicgoldprospectingforum.com. All messages on this forum express the personal views of the author and should not be interpreted as necessarily being in accord with those of the forum owner and neither the owner of this forum and its domain name nor SMF or the forum software developers or the forum host shall be held responsible for the content of any message. Admin reserves the right to remove any offensive or objectionable posts. No defamatory material or politics/religion or issues of race will be permitted.
QED coming soon!
Goldsearch Australia has been appointed the Australian Distributor for the QED.Check their website after  Wed 7th Dec 2016.
Global temperature goes from heat record to heat record, yet the sun is at its dimmest for half a century.
What is empirical  evidence?
Empirical evidence is information acquired by observation or experimentation. This data is recorded and analyzed by scientists and is a central process as part of the scientific method. Why is everything posted on climate by Inhere always horseshit?
Poor old  Inhere on finders still does not know the difference between weather and climate!The silly old goat seems to think because some places have recently had very cold weather that this brings into question  AGW!  Only right wing rabble rubbish climate deniers like horseshit Inhere don't  seem to understand  that AGW will mean that some regions of the earth will become colder!
Why is  onehalfgram :1/2 wit always wrong  on the QED?

australian electronic gold prospecting forum.com  |  Notices from the owner and Administrator  |  Admin notices and forum guidelines  |  Topic: Wayback machine archiver is now being blocked from accessing this forum 0 Members and 1 Guest are viewing this topic. « previous next »
Pages: [1] Go Down Print
Author Topic: Wayback machine archiver is now being blocked from accessing this forum  (Read 298 times)
Doug
Administrator
Revered Supreme Hero Member
*******
Offline Offline

Posts: 14914



« on: Monday July 27 2015 19:05:40 AEST PM »

The experimental archiver wayback machine is now being blocked from accessing this forum.It ignores the robots txt file in the public html folder of the  forum server despite saying that it will respect it.Many other archivers are also now being blocked as well as they consume bandwidth which i pay for!
doug smile
Logged

All posts on this forum are the personal views of the author and should  not necessarily be  interpreted as those of Admin The QED contains NO patented or protected IP!!! No fake users on this site! This forum does not depend on  guest posting liars to survive!1/2 wit powerless to login and post! LO
Doug
Administrator
Revered Supreme Hero Member
*******
Offline Offline

Posts: 14914



« Reply #1 on: Tuesday July 28 2015 12:59:22 AEST PM »

  
The experimental archiver wayback machine is now being blocked from accessing this forum.It ignores the robots txt file in the public html folder of the  forum server despite saying that it will respect it.Many other archivers are also now being blocked as well as they consume bandwidth which i pay for!
doug smile

The server log files show that the archiver is still trying trying to access the forum but gets an error 403 access denied message. It should be noted that archivers can only access boards that are  accessible  by guests.
I have also  implemented this:
To remove a site from the Wayback Machine, place a robots.txt file at the top level of your site (e.g.   ) and then submit your site below.
The robots.txt file will do two things:

     1.   It will remove all documents from your domain from the Wayback Machine.

     2.   It will tell the Internet Archive's crawler not to crawl your site in the future.

To exclude the Internet Archive's crawler (and remove documents from the Wayback Machine) while allowing all other robots to crawl your site, your robots.txt file should say:

                       User-agent: ia_archiver

                       Disallow: /

Robots.txt is the most widely used method for controlling the behavior of automated robots on your site (all major robots, including those of Google, Alta Vista, etc. respect these exclusions). It can be used to block access to the whole domain, or any file or directory within. There are a large number of resources for webmasters and site owners describing this method and how to use it.
doug smile
Logged

All posts on this forum are the personal views of the author and should  not necessarily be  interpreted as those of Admin The QED contains NO patented or protected IP!!! No fake users on this site! This forum does not depend on  guest posting liars to survive!1/2 wit powerless to login and post! LO
Doug
Administrator
Revered Supreme Hero Member
*******
Offline Offline

Posts: 14914



« Reply #2 on: Tuesday July 28 2015 17:34:44 AEST PM »

  
  
The experimental archiver wayback machine is now being blocked from accessing this forum.It ignores the robots txt file in the public html folder of the  forum server despite saying that it will respect it.Many other archivers are also now being blocked as well as they consume bandwidth which i pay for!
doug smile

The server log files show that the archiver is still trying trying to access the forum but gets an error 403 access denied message. It should be noted that archivers can only access boards that are  accessible  by guests.
I have also  implemented this:
To remove a site from the Wayback Machine, place a robots.txt file at the top level of your site (e.g.   ) and then submit your site below.
The robots.txt file will do two things:

     1.   It will remove all documents from your domain from the Wayback Machine.

     2.   It will tell the Internet Archive's crawler not to crawl your site in the future.

To exclude the Internet Archive's crawler (and remove documents from the Wayback Machine) while allowing all other robots to crawl your site, your robots.txt file should say:

                       User-agent: ia_archiver

                       Disallow: /

Robots.txt is the most widely used method for controlling the behavior of automated robots on your site (all major robots, including those of Google, Alta Vista, etc. respect these exclusions). It can be used to block access to the whole domain, or any file or directory within. There are a large number of resources for webmasters and site owners describing this method and how to use it.
doug smile

Here is an example of wayback archiver being blocked! error 403 access denied!
doug smile


* blocked archiver.jpg (180.72 KB, 699x633 - viewed 65 times.)
Logged

All posts on this forum are the personal views of the author and should  not necessarily be  interpreted as those of Admin The QED contains NO patented or protected IP!!! No fake users on this site! This forum does not depend on  guest posting liars to survive!1/2 wit powerless to login and post! LO
Pages: [1] Go Up Print 
australian electronic gold prospecting forum.com  |  Notices from the owner and Administrator  |  Admin notices and forum guidelines  |  Topic: Wayback machine archiver is now being blocked from accessing this forum « previous next »
Jump to:  


Login with username, password and session length

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2006-2009, Simple Machines
Simple Audio Video Embedder

BisdakworldClassic design by JV PACO-IN
Valid XHTML 1.0! Valid CSS!
gold