ARM CPUs for power efficient Web Servers - James agrees...

James Hamilton highlights the power efficiency of ARM for general purpose web servers. It's almost two years since I gave the first talk on Millicomputing at the HPTS Workshop (which is where I met James for the first time) so its great to see him talking up the principles. He also makes the important point that ARM uses error correcting (ECC) memory, while the Intel Atom doesn't, and thus the Atom is actually less suitable for configuring large numbers in low power enterprise server applications.

The systems at http://www.linux-arm.org/Main/LinuxArmOrg are relatively inefficient blades, they have archaic spinning rust storage attached which must dominate the power consumption. A flash based storage subsystem would make much more sense to me. Web content delivery workloads are very well suited to low cost read-mostly flash storage. They do have a 1.2GHz ARM CPU and 1.5GB of RAM per blade, which is the biggest and fastest ARM configuration I've seen so far.

Lower Power x86 Systems

There is quite a lot of activity in the low power x86 compatible space. The latest CPU from VIA is touted as a 1W CPU, with 0.1W standby power, but when the complete chipset and RAM are added its substantially higher, more like 10W. This article in LinuxDevices.com surveys the whole space very nicely.

The power trend is downwards, these chip sets are aimed at consumer devices, but not in the battery powered space. For commodity devices we can divide by the order of magnitude in power consumption and environmental conditions.

100-1000W Datacenter Server (air conditioned room)

10-100W Home PC/Laptop Space (fan cooled, on when in use, ambient room temp)

1-10W Home consumer devices (fanless, always on, ambient room temp) 

100-1000 mW Battery powered millicomputers (always on, cool enough for your pocket)

Interesting technology and product disruptions occur when we mix these spaces. In some ways, the original compute farms that Google built were leveraging the low end home PC power/price/performance point into the datacenter. There are additional opportunities to leverage home consumer devices and millicomputers into the enterprise space.

Enterprise Millicluster

Taking the components I have already menioned in previous posts, we can assemble them into a small cluster that seems to be a useful size and specification for an Enterprise Server building block. Its a cluster of millicomputers, so we may as well coin the name Millicluster as we go along (and register the millicluster.com etc. domains to point here :-)

Using 8-port USB switches, we could lay out 14 i.MX31 based Millicomputer modules behind a PPC440EPx based Ethernet Bridge that runs Linux so it is general purpose, but it will be pre-configured as a Load Balancer. This gives us a 1 Gbit/sec redundant network (it has two 1 Gbit links in, but only two 480Mbit links to the millicomputer modules). There is a total of 7.5 GHz of CPU, 3.5 GBytes of RAM, and 56 Gbytes of Storage using 4 GByte microSDHC flash memory cards on each Millicomputer.

This depends upon having a high speed 8-port switch, and so far I have found some products from Belkin and D-link that have one upstream port and seven downstream. I'm not sure what chipset they use, but they are inexpensive and have been available for a few years, so this seems reasonable.

microSD and microSDHC Storage for Millicomputers

Its been a while since I last posted here, mostly due to my change in employer, as mentioned on my personal blog... Now that is settled, I have a backlog of things to discuss here. Also, I'll be at the Techshop pavillion at Maker Faire this Saturday and Sunday afternoons.

There is an interesting price graph for 2GByte microSD at Nextag. It shows an introduction at about $120 last September, trending down to about $25 now. Amazon has it for $25 and Amazon Marketplace for $16.50. Meanwhile, larger devices are shipping using the microSDHC format at 4GByte, and announcements have been made for 8GByte. The microSD format gives a performance of around 20MBytes/s, so for 2KB reads that works out at a maximum of 10,000 random reads/second. Writing at 6MB/s works out at a maximum of 3000 random 2KB writes/s. Even allowing for operating system overhead, several thousand IOPS should be possible at under ten cents per IO per second and $10/GByte.

Of course, most millicomputer CPUs interface directly to devices like microSD, so there is no support device or interface needed. Just a microSD carrier or two, per millicomputer module.

Millicomputer Based Load Balancers

If we build systems that contain hundreds of modules for web based applications, we need a way to manage the workflow distribution for incoming network traffic. Commercial load balancers cost more than millicomputing modules we want to send load to, so I've been looking around for open source projects that implement various kinds of load balancing. I have found a very good detailed summary article on this subject by Willy Tarreau, author of HAproxy, which he describes as:

HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for web sites crawling under very high loads while needing persistence or Layer7 processing. Supporting tens of thousands of connections is clearly realistic with today's hardware. Its mode of operation makes its integration into existing architectures very easy and riskless, while still offering the possibility not to expose fragile web servers to the Net...

At the http/application level, I found a description of a simple but powerful tool called balance.

Balance is our surprisingly successful load balancing solution being a simple but powerful generic tcp proxy with round robin load balancing and failover mechanisms. Its behaviour can be controlled at runtime using a simple command line syntax.

Another http load balancer that claims high performance and more features is XLB. It states

XLB is a high performance HTTP load balancer. connection management, caching, ssl, scripting. 300 mbit/sec / 4000 reqs/sec takes 30% cpu on a 2GhZ Xeon. connection pooling to backend servers reduces memory and cpu usage on backends.

One problem with load balancers, is that if they fail, a potentially large number of modules would be out of action. The Ultra Monkey load balancer addresses this issue.

Ultra Monkey 3 makes use of The Linux Virtual Server (LVS) to provide fast load balancing. The Linux-HA framework is used to monitor the linux-directors - the hosts running LVS and doing the load balancing. This is combined with ldirectord which monitors real-server - the hosts that accept end-user's connections. These three core components allow Ultra Monkey 3 to provide highly available and/or load balanced network services.

I haven't used any of these options, so I'm very interested to get recommendations, please comment if you have experience or alternatives to share, and I'll update this post.

In the array of modules scenario, I would dedicate a few modules to provide load balancing services. If the modules are all connected via Ethernet, then any module can be used. If we use the USB network then the central USB master that provides an Ethernet gateway is the natural place to install load balancer services.

Vertical and Horizontal Module Arrangements

Modules are available with edge connectors that can be mounted in bulk on a mother board as shown in the image below. The dimensions match the standard motherboard found in 1U Enterprise server designs, about 12x13 inches. The diagram shows 120 modules, but its quite likely to be possible to pack them in more densely than this.

The alternative is to mount modules flat on the boards as shown in the second diagram. This has the same 12x13 inch area, but is a very thin board, and at least four of them could be stacked in a 1U package, which also comes out to 120 modules.





In practice these board sizes and layouts will need to be adjusted to take into account the mechanical problems of flexing, mounting, cable routing etc. In each case the power and cooling management should be relatively simple, since there is a total peak power of around 100 watts for the entire 1U package, and no localized hot spots.

Some of the module designs have built-in temperature sensors and they all have power voltage sensors, so they can detect and report on environmental conditions across the motherboard.

Millicomputer Module Interconnects

There are two basic approaches.

One is to get modules that have ethernet built-in (or to add ethernet interfaces to a motherboard) and use ethernet switch chips such as the 8-24 port solutions from Vitesse to cluster the modules together. The individual modules would connect at 100Mbit, and the switches and external interfaces would interconnect at 1Gbit. The single chip ethernet switches have lots of features but can be run as unmanaged devices, so there is very little software needed to implement or manage the network. By directly connecting the networks on a motherboard there is no need to drive the full physical ethernet wire standard between the devices, saving a lot of power. These devices cost a few dollars a port, and dissipate about half a watt per port for fully driven gigabit links. if we can avoid using the Ethernet "PHY" (physical driver) a lot more power can be saved.

Another option is to use the built-in high speed USB2.0 interfaces which run at up to 480Mbit/s and connect them to a USB based central router that has ethernet support, then run IP over USB. This is a bit more complex to implement, but could be faster, lower power and cheaper since it uses an interface that is directly built into the millicomputer CPU. There are other kinds of devices like the AMCC PPC440EPx that are more PC-like, and have ethernet, PCI-bus and high speed USB built-in that could be used to implement a board level controller/router/interface. This device is more powerful than the mobile oriented millicomputer CPUs but dissipates about 3W so its in the next bracket up from a power consumption viewpoint.

PXA270 Module for testing

I just ordered a Gumstix GS270-XL6P module with 600MHz PXA270 and 128MB RAM. I'll run benchmarks on it, then end up building it into one of the mobile phone designs I'm working on. More later...

Millicomputer Module Specifications

Here is another Google spreadsheet table of millicomputing module specifications.

There are several approaches, but some of these are edge connector based, include on-board ethernet, and could be stacked on a motherboard in a very dense array.

I think that on a standard 1U motherboard, if we could get five rows of 24 connectors that is 120 individual modules, using less than 100W maximum. The motherboard would just need to provide power and ethernet switch chips. If we also want per node storage, there are many very dense NAND flash ships in the multi-Gigabyte range that could be added to the design.

So is that interesting? I think so...

Millicomputer CPU Specifications

I've started a table of CPU specifications as a google spreadsheet.

I'm mostly interested in the CPU clock rate, CPU caches, RAM bandwidth and size.

All these devices are very flexible, and are mostly configured with relatively small amounts of memory for embedded applications. However, they have a decently fast clock rate, and can interface to at least two SDRAM chips. These chips are 32bits wide and currently contain a total of 128MB each. The CPUs support up to 256MB per chip, so the next generation SDRAM devices can double overall capacity.

Compared to current "enterprise CPUs" they are much slower than Opterons but probably comparable to a single thread on a Niagara.

The next comparison table I want to put together is for board level devices, such as the Gumstix range.

Flash Solid State Disk (SSD) for Millicomputers

The days of keeping bits on spinning rust are coming to an end....

Both Samsung and SANdisk have announced 32GB SSDs. SANdisk's comsumes 0.9W max (competing disks take 1.9W) and fits in a 1.8 or 2.5" drive form factor with ATA interface. The sequential performance of these SSDs is similar to normal disks for reads, a bit slower for pure writes, but as soon as you start doing random reads or writes they are an order of magnitude faster than disks. The smaller the random accesses the bigger the relative speedup. The latest announcement from Samsung is a 1.8" 64GB version, and there is some discussion about the growth of this market in the press release.

This makes perfect sense for millcomputers. Small millicomputers can be directly connected to gigabytes of NAND flash via the SDIO interface, and larger millicomputers can use ATA interfaces to connect to flash-SSDs. The extra random performance of the SSD offsets the lack of disk spindles in a compact design and will make IO intensive workloads extremely competitive for millicomputing.

The MTBF (reliability) of SSDs is also far higher than disks. A mirrored pair of disks may be replaced with a single SSD since it has much higher reliability. This helps offset the current price premium paid for the SSD.

In the past SSDs have been built using technologies that were far more expensive than disks. Flash based SSDs have now reduced the gap, and the trend is that SSDs will eventually become bigger and cheaper than disks, the only question is when, and my answer is sooner than you think!

Update: here is a detailed benchmark review from Tomshardware.com.

What is a Millicomputer? Why talk about Millicomputing?

While researching devices for my home brew mobile phone, I've realized that the current generation of CPUs for mobile devices are actually seriously powerful, very low cost and use almost no power. The performance per watt and per dollar seems to be an order of magnitude better than the PC-class CPUs that are common in commodity servers nowadays. The absolute performance and memory capacity is lower, but is comparable to common PC hardware from a few years ago, and could be useful for more than running a high end phone or portable games machine. Devices such as the Marvel PXA270 and Freescale i.MX31 run at over 500MHz, some include floating point units, they support at least 128MB of RAM (a single chip), and a myriad of I/O interfaces, with Linux 2.6 support.

While the current mainstream CPUs were driven by the development of the home PC market, this generation is driven by the development of the mobile, battery powered device market, which is a very large. For example the worldwide cellphone market is something like a billion devices a year.

I think that there could be some interesting general purpose computer systems built from low power devices (CPUs that use less than one watt). I looked around but wasn't sure what to search for... I do know about the systems that are sold for embedded use, but they are typically configured using lower speed and lower memory options.

Does anyone know of vendors selling general purpose millicomputer based systems?

I need a name for this class of system, so I'm going to call them Millicomputers, and I'm going to explore this area in public on this blog, and using the principles of open hardware that we have adopted for the homebrew mobile phone club, I expect to help build some.

I originally asked this question on my main blog, and asked a lot of people in person, but didn't find a pre-existing name or any objections to this concept.