I've been doing a LOT of testing and research on this and to the best of my findings the server response time is most closely related to the number of CPU's and amount of RAM. I tested on traditional Reseller server provided by Site5 which I believe is 1 CPU and 1Gb RAM and the best I could get was 250 Milliseconds. When I tested the same thing on DigitalOcean's 512Mb with 1 CPU and a SSD I got about the same. Yet when I upgraded to 8Gb 4 CPU I got under that 200 milliseconds.
My advice is to have a SSD with 1Gb or RAM or More and 2 CPU's. Or a 2Gb 2 CPU HDD.