There are two basic approaches.
One is to get modules that have ethernet built-in (or to add ethernet interfaces to a motherboard) and use ethernet switch chips such as the 8-24 port solutions from Vitesse to cluster the modules together. The individual modules would connect at 100Mbit, and the switches and external interfaces would interconnect at 1Gbit. The single chip ethernet switches have lots of features but can be run as unmanaged devices, so there is very little software needed to implement or manage the network. By directly connecting the networks on a motherboard there is no need to drive the full physical ethernet wire standard between the devices, saving a lot of power. These devices cost a few dollars a port, and dissipate about half a watt per port for fully driven gigabit links. if we can avoid using the Ethernet "PHY" (physical driver) a lot more power can be saved.
Another option is to use the built-in high speed USB2.0 interfaces which run at up to 480Mbit/s and connect them to a USB based central router that has ethernet support, then run IP over USB. This is a bit more complex to implement, but could be faster, lower power and cheaper since it uses an interface that is directly built into the millicomputer CPU. There are other kinds of devices like the AMCC PPC440EPx that are more PC-like, and have ethernet, PCI-bus and high speed USB built-in that could be used to implement a board level controller/router/interface. This device is more powerful than the mobile oriented millicomputer CPUs but dissipates about 3W so its in the next bracket up from a power consumption viewpoint.