HOMENEWSCautions for accessing external servers from ABCI
Cautions for accessing external servers from ABCI
Frequent access to external servers by crawling from ABCI to retrieve data sets may cause significant inconvenience to the sites from which the data is collected.
The appropriate frequency of access depends on the server performance and capacity of the target site, but as a general rule, we ask that you control access to the same site no more than once per second in a row.
This does not preclude the use of external data acquisition methods such as crawling.