I’m glad when people find our web sites and enjoy them. I guess the whole purpose of putting anything up on the web is to share. We don’t do anything special to try and drive traffic to our sites, because we figure people who want to find the stuff we have, will. And I have nothing against being listed in search engines – that’s a good thing, really – LOL. I have mostly thought of the individual people who access our sites instead of ‘organizations’.
I guess I’m just a bit conflicted however, that we’re paying for the bandwidth for someone like, for example (not picking on them in particular – they’re just the most recent in our referrer logs), www.ja.net, that is accessing all of the pages on the ‘cc’ site. They are the “United Kingdom Education & Research Networking Association” that says this about themselves, “JANET is connected to the equivalent academic networks in other countries and to many commercial networks in the UK and abroad forming part of the global internet.” They also say that they are government funded and it seems they allow use of their services only by “Higher Education Institutions or Research Councils”. Yes, on the one hand its interesting that ja.net considers our site worth indexing, but… *I* wouldn’t be eligible to use their site… ????
Anyway… I’m not spending sleepless nights thinking about this – LOL – but it WOULD be nice if places like that followed the Robots Exclusion Standards like other search engines do. I suppose this is just encouraging me to create an active “mod_access” ability on our server so I can “…restrict access based on the hostname or IP address from which the user is connecting to your site.” I guess when all is said and done the whole ISSUE is just about consciously allocating the use of our monthly allowed bandwidth. Exciting, no? (That was rhetorical!)