It's not the route I've gone, but lots of people seem to be having success with retired small form factor business machines. Getting a good deal on used business stuff seems to be the key, ideally locally, where they just want to get rid of it compared to ebay. A low-end Intel from 4 years ago is annoyingly good computation per watt.
Also, take your power bill and calculate the cost of a watt for a year. Remember that number for all future purchases. Where I live, it's around $1.50/watt-year. A 60-watt always-on computer gets pricy quickly.