# We don't want web "robots" to index the guts of postscript or binary files. # It wastes bandwidth. They are large and not indexable. # Don't even let them try protected or temporary areas User-agent: * Disallow: /gross/thesis # Contains only postscript; robots seem to know about .gz Disallow: /gross/CfAOnly Disallow: /gross/ICASEOnly Disallow: /gross/SantaCruzCosmoOnly Disallow: /gross/USGSOnly Disallow: /gross/UWAstroOnly Disallow: /gross/temp