Millicomputing Applications - ETL

Millicomputers have a very different balance of compute/memory/network/io resources compared to more conventional architectures. They are lower in absolute terms for compute/memory/network, but much higher for random access io.

The performance per watt and the price/performance are very competitive for compute/memory/network, as long as applications can be run at a smaller "grain size". However for io, the aggregate performance of a large number of direct attached flash devices is amazing.

One possible application is from the data warehousing space. Known as ETL, this is the Extract Transform and Load step that pulls data from online transaction processing systems, such as the collection of database back-ends for a web site, and puts it into a form that can be queried to answer questions about the business. There has been a lot of work put into making the ETL processes into decomposable parallel applications, and there is an open source ETL implementation in Java called KETL. KETL was originally written several years ago, when the typical systems of the day were similar in capacity to the millicomputers we have today, so I'm hopeful that the grain size will fit.

KETL is io intensive and it also supports running on a cluster of networked computers, so overall it looks like a plausible fit for an enterprise millicomputer application.

The Future of Millicomputing

There is a gap between the performance and memory capacity of Millicomputers and mainstream CPUs, that gap is shrinking but how fast, and what are the next steps?

The base technology from ARM can be seen in their Cortex designs. These were disclosed in late 2005, but have yet to appear in actual products. The overall performance is around 3-4 times the performance of the current generation of ARM based devices.

Since Intel sold off their ARM based CPU business to Marvell, it leaves them clear to move their core 32bit x86 platform architecture down into the millicomputing space.

So in the next few years, I expect to see x86 and ARM based system on a chip architectures with overlapping performance and power consumption characteristics in the millicomputing space.

Lower Power x86 Systems

There is quite a lot of activity in the low power x86 compatible space. The latest CPU from VIA is touted as a 1W CPU, with 0.1W standby power, but when the complete chipset and RAM are added its substantially higher, more like 10W. This article in LinuxDevices.com surveys the whole space very nicely.

The power trend is downwards, these chip sets are aimed at consumer devices, but not in the battery powered space. For commodity devices we can divide by the order of magnitude in power consumption and environmental conditions.

100-1000W Datacenter Server (air conditioned room)

10-100W Home PC/Laptop Space (fan cooled, on when in use, ambient room temp)

1-10W Home consumer devices (fanless, always on, ambient room temp) 

100-1000 mW Battery powered millicomputers (always on, cool enough for your pocket)

Interesting technology and product disruptions occur when we mix these spaces. In some ways, the original compute farms that Google built were leveraging the low end home PC power/price/performance point into the datacenter. There are additional opportunities to leverage home consumer devices and millicomputers into the enterprise space.