Hey guys,. It's been a while! Any rules of thumb as to exactly what are the minimum directories that need to be made open in robots.txt so ...
Is there any way with PW to do environment-specific robots.txt, i.e. to block robots from staging sites without having to manually edit ...
Old Hard to Find TV Series on DVD
I tried with robots.txt. but this crawler does not seem to respect what is configured. in robots.txt. I need something more powerfull with a ...
I have a site that has other ccTLDs, such as .co.za, .co.il, etc., I am trying to make a robots.txt for each of the different domains by following this article.
Use a robots.txt or the meta tags as you describe. I prefer option 1 since it's a matter of keeping everyone out except clients with an username ...
Hi there,. i developed a single page site where the sections of the site are pages within processwire. For example imagine this case: I have ...
What about the wonderfull ProcessWire CMS? Anybody tried to install via git clone of the official repository?
I have installed processwire on 2 of my domains which are hosted by different hosting company's but both of them fail to give me the admin page.
I want to allow clients to login but I assume from a security POV it's better to not have a link to, say, /processwire/ in the footer as doing ...
The robots.txt file disallows any directories and files that automated bots should not crawl ... Do you have more general feedback about our service? Please use ...