element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products and Partners
    Products and Partners
    • Arduino
    • Avnet & Tria Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • About Us
    About the element14 Community
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Japan
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      •  Vietnam
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
element14's The Ben Heck Show
  • Challenges & Projects
  • element14 presents
  • element14's The Ben Heck Show
  • More
  • Cancel
element14's The Ben Heck Show
Forum Build Challenge --- Super-Computer
  • Blog
  • Forum
  • Documents
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join element14's The Ben Heck Show to participate - click to join for free!
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 6 replies
  • Subscribers 29 subscribers
  • Views 695 views
  • Users 0 members are here
Related

Build Challenge --- Super-Computer

e14 Contributor
e14 Contributor over 15 years ago

Super computers have been around for a long time.  But unless you've got a couple million dollars in your pocket, you're SOL (Simply Out of Luck). 

 

 

Or are you?

 

 

A recent trend in super computing is to link multiple, off-the-shelf computers into a cluster and use free software to pool all available resources and make it act as one big computer. Of course, they usually come out looking like a lexan delivery truck that got into a wreck with a motherboard delivery van.  But for the researcher on a budget, it's the cheapest way to go.

 

 

The Challenge:

 

 

To design a sleek looking setup that can fit on your desk without taking up the whole surface using a router.

 

 

There is one caveat; Gigabit ethernet is slow.

 

 

I know, I know; it's the greatest thing since sliced bread.  1000 bits a second, it slices, it dices, and leaves the competition in the dust.  To think of improving on the perfection that is this technological Eigth Wonder of the World is MADNESS!

 

 

But there is something better; USB 3.0.  It has a max capacity of 5 GB/s before overhead, and 3.4 after over head.  It costs much less than gigabit ethernet cables.   And I believe if you could combine the MAC addressing system of gigabit ethernet with the bandwidth of USB 3.0 and engineer a USB Router-Switch to link up multiple motherboards, then the end-result would be a 3.4 times faster than ethernet, maybe more because you might be able to decrease the overhead that the USB 3.0 bridge would need to run multiple devices.  What this means is that one motherboard could talk to another, sharing information at transfer rates that are as high if not higher than components on an individual motherboard could (such as the bus that handles CPU and RAM transfers).  Which would translate to more FLOPS a second.  Which would mean more bang for the buck on a researchers already anorexic budget. Not only that, but a device like this could potentially take the place of gigabit ethernet routers in the home.

 

 

There are a couple small-form factor motherboards on the market incorporating USB 3.0 into them, and as the demand for USB 3.0 increases, more are bound to hit the market.  There is a site or two dedicated to the building of small-scale super-computer clusters, and if you want information regarding this build, feel free to contact me.  A great way to increase the flexibility of this system is by dispensing with simple "hubs" that can run fewer than 3 devices at once and convert them into full-fledged switches.

 

That is my build challenge, Mr. Heck;  design a MAC addressing system for a USB 3.0 hub, plug it into an incredibly compact multi-motherboard super-computing, Crysis playing network cluster, complete with all the fancy doo-dads, knick-knacks, and thing-a-ma-bobs that make computers awesome.  It requires more knowledge than the lowly intro-level engineering student that I am possesses to accomplish, or otherwise, we would be having a very different discussion in a forum labeled "Show off your build."  I think that this build, should you choose to accept it, could fill a vital role in the market, both the niche market of super-computer cluster building as well as the commercial one of home-router systems and make USB an even more useful technology than it already is.

 

 

And maybe, just maybe, my children would grow up in a world where they could play Crysis on max settings for under $1200.

  • Sign in to reply
  • Cancel
Parents
  • e14 Contributor
    e14 Contributor over 15 years ago

    I understand the speed difference in USB 3.0.. but there are multi gigabit ethernet things.. its called Fiber Optics.. however.. there is already a supercomputer for the budget. Check out Nvidia Tesla. using the extreme power of the GPU er.. multiple GPUs you can clearly get high performance supercomputing desktop..

    my GPU (I currently have 2) is runing at just over 600 MHZ this does not seem very much because because my processor is running at 3333 MHZ and that is the transfer rate of USB 3.0..

    here is why you may be wrong.

    at 602 MHZ I am using 240 Processing cores PER GPU. Again, I have 2 GPUs

    A tesla system can have 4 GPUs more powerful than mine

    http://www.supermicro.com/products/system/4U/7046/SYS-7046GT-TRF.cfm?GPU=FC407 <<< check out this system

    Supermicro has many suprecomputer systems tethered via gigabit Ethernet, (I saw them when I was in the navy) BUT.. somehow the motherboards the military choose did were not reliable and had to be replaced... alot..

    but.. don't forget the data transfer rate of memory "unless you have DDR3" is fairly limited

    my final thought is...  with a new Nvidia GTX 560 or 570 kids can in fact play crysis on max for around $1200

    go to Ibuypower.com and see for youself

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • e14 Contributor
    e14 Contributor over 15 years ago in reply to e14 Contributor

    But fiber optics, generally speaking, isn't a household consumer item that "needs inventing" like a USB 3.0 switch could be, which is one of the things that Ben Heck puts down in his blog as a sort of  "pre-requisite" if you will for submitting ideas.  Although at roughly $800 for 1000 ft. it could work --- if you can afford fiber-optic router price ranges from $500 - $5,000.  A gigabit ethernet router costs around $160; USB 3.0, cheaper, but, heck, let's say it's not even 25% cheaper, could be $120 if done right.

     

    I couldn't even find a price online for the Tesla short of calling someone up at nVidia, which suggests that this is not a normally stocked "off-the-shelf" item, and even if it were, you might not be able to get one as cheaply as you could build, say, a four-node computer cluster running theoretically on USB 3.0 with a USB 3.0 switch.  There is a site http://www.clustermonkey.net//content/view/211/1/ that throws out the parts list for a four node cluster as I have described (minus the USB stuff) for $1250 (accounting for decrease in parts cost).  Add on a couple mid-level GPU's to each motherboard, you're looking at $1800, no harm no foul.  My entire setup (including an ATI Radeon 5850, new power-supply, and 23" flat-screen monitor) cost me $1200 bucks. 

     

    In reference to your "multiple GPU" theory, there is a slight snag; running multiple GPU's in the same box can increase performance --- on programs that have been created to utilize those extra resources.

     

    On most programs, they only use one or the other GPU, and while you are correct in asserting their dominion over the super-computing scene, there are a couple of programs available for free ( http://www.open-mpi.org/ and http://www.csm.ornl.gov/pvm/ ; bear in mind one or the other of these programs would need to be modified to call upon the USB bridgework, as they are currently adapted to gigabit ethernet alone ) that could effectively combine the resources (GPU's on each motherboard included) at a level that is above the hardware but below the program; as such any program run on this machine (this is an assumption, and we all know what happens to "u-and-me" when we assume) would run off of this collective pool of resources and, with a USB 3.0 setup, transmit data from the head node to the slave nodes much more effectively than in a system running gigabit ethernet.

     

    AND, the point of the MAC addressing system is to ELIMINATE OVERHEAD that the USB 3.0 system uses, which would make for a theoretical amount of 5 GB/s (Might be less; let's be honest, nothing's perfect, and even if it were, it might still require SOME overhead), which is 1.6 GB's faster than what you're running at.  And, the supercomputing cluster advantage over your specific system (not including the system you linked, although I'll address my grievance about that in a moment) is that it has multiple motherboards, which could each contain multiple GPU's, and multiple processors with multiple cores which could all, if you were so inclined, run at 3.33 GHZ.  Four horses would pull harder than just the one, not-withstanding any GPU advantage/disadvantage.

     

    As for the system you mention, I drooled when I looked at it.  However, it, much like the nVidia Tesla, is curiously devoid of a price-tag at first blush.  Perhaps someone has keener eyes than mine and can find out how much something like that would go for.  But again I suspect it might be a smidge higher than the $1250 computer cluster in the aforementioned link.  And again, four nVidia Tesla's running on gigabit ethernet (as the site suggests this system uses to link each of those "blades") would be fast, but not as fast as this same system with a USB 3.0 framework as I describe running the exact same hardware.

     

    As for the limitations of data transfer rates for DDR3 memory and lower, you are, again, correct.  Even with DDR3 or higher, there is no assurance that the system can utilize this extra bandwidth due to Front-side Bus constraints.  However, Intel and AMD have both been researching their corporate posteriors off with their ( http://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect ) and  ( http://en.wikipedia.org/wiki/HyperTransport ) technologies, respectively.  Hopefully they can improve FSB speeds in the same way USB speeds have been improved.  But one step at a time; get one or the other in FIRST, then do the other one.

     

    To your final thought I say BAH!

    ( http://www.amd.com/us/products/desktop/graphics/ati-radeon-hd-5000/hd-5850/Pages/ati-radeon-hd-5850-overview.aspx )  Auto-detect in Crysis gives me a High setting with a Dell Studio XPS 8000 with no modding aside from said card and bigger power-supply.  But let's not get into the "my <insert brand> is better than your <insert brand>" debate because it's pointless.  Suffice it to say---I muddle through. image

     

    However, NONE of this conjecture, while fun and entertaining, may out do THIS technology ( http://en.wikipedia.org/wiki/Light_Peak ) in the relatively near future.

     

    ***Somehow, I am not surprised you were in the Navy --- most, if not all, the coolest entrepreneurial kinds of people were in some way, shape, or form militarily involved.  Me, I was just a military brat; close enough to no longer fit in with people, but far enough away to not stand at attention when my SO walks into the room. 

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
Reply
  • e14 Contributor
    e14 Contributor over 15 years ago in reply to e14 Contributor

    But fiber optics, generally speaking, isn't a household consumer item that "needs inventing" like a USB 3.0 switch could be, which is one of the things that Ben Heck puts down in his blog as a sort of  "pre-requisite" if you will for submitting ideas.  Although at roughly $800 for 1000 ft. it could work --- if you can afford fiber-optic router price ranges from $500 - $5,000.  A gigabit ethernet router costs around $160; USB 3.0, cheaper, but, heck, let's say it's not even 25% cheaper, could be $120 if done right.

     

    I couldn't even find a price online for the Tesla short of calling someone up at nVidia, which suggests that this is not a normally stocked "off-the-shelf" item, and even if it were, you might not be able to get one as cheaply as you could build, say, a four-node computer cluster running theoretically on USB 3.0 with a USB 3.0 switch.  There is a site http://www.clustermonkey.net//content/view/211/1/ that throws out the parts list for a four node cluster as I have described (minus the USB stuff) for $1250 (accounting for decrease in parts cost).  Add on a couple mid-level GPU's to each motherboard, you're looking at $1800, no harm no foul.  My entire setup (including an ATI Radeon 5850, new power-supply, and 23" flat-screen monitor) cost me $1200 bucks. 

     

    In reference to your "multiple GPU" theory, there is a slight snag; running multiple GPU's in the same box can increase performance --- on programs that have been created to utilize those extra resources.

     

    On most programs, they only use one or the other GPU, and while you are correct in asserting their dominion over the super-computing scene, there are a couple of programs available for free ( http://www.open-mpi.org/ and http://www.csm.ornl.gov/pvm/ ; bear in mind one or the other of these programs would need to be modified to call upon the USB bridgework, as they are currently adapted to gigabit ethernet alone ) that could effectively combine the resources (GPU's on each motherboard included) at a level that is above the hardware but below the program; as such any program run on this machine (this is an assumption, and we all know what happens to "u-and-me" when we assume) would run off of this collective pool of resources and, with a USB 3.0 setup, transmit data from the head node to the slave nodes much more effectively than in a system running gigabit ethernet.

     

    AND, the point of the MAC addressing system is to ELIMINATE OVERHEAD that the USB 3.0 system uses, which would make for a theoretical amount of 5 GB/s (Might be less; let's be honest, nothing's perfect, and even if it were, it might still require SOME overhead), which is 1.6 GB's faster than what you're running at.  And, the supercomputing cluster advantage over your specific system (not including the system you linked, although I'll address my grievance about that in a moment) is that it has multiple motherboards, which could each contain multiple GPU's, and multiple processors with multiple cores which could all, if you were so inclined, run at 3.33 GHZ.  Four horses would pull harder than just the one, not-withstanding any GPU advantage/disadvantage.

     

    As for the system you mention, I drooled when I looked at it.  However, it, much like the nVidia Tesla, is curiously devoid of a price-tag at first blush.  Perhaps someone has keener eyes than mine and can find out how much something like that would go for.  But again I suspect it might be a smidge higher than the $1250 computer cluster in the aforementioned link.  And again, four nVidia Tesla's running on gigabit ethernet (as the site suggests this system uses to link each of those "blades") would be fast, but not as fast as this same system with a USB 3.0 framework as I describe running the exact same hardware.

     

    As for the limitations of data transfer rates for DDR3 memory and lower, you are, again, correct.  Even with DDR3 or higher, there is no assurance that the system can utilize this extra bandwidth due to Front-side Bus constraints.  However, Intel and AMD have both been researching their corporate posteriors off with their ( http://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect ) and  ( http://en.wikipedia.org/wiki/HyperTransport ) technologies, respectively.  Hopefully they can improve FSB speeds in the same way USB speeds have been improved.  But one step at a time; get one or the other in FIRST, then do the other one.

     

    To your final thought I say BAH!

    ( http://www.amd.com/us/products/desktop/graphics/ati-radeon-hd-5000/hd-5850/Pages/ati-radeon-hd-5850-overview.aspx )  Auto-detect in Crysis gives me a High setting with a Dell Studio XPS 8000 with no modding aside from said card and bigger power-supply.  But let's not get into the "my <insert brand> is better than your <insert brand>" debate because it's pointless.  Suffice it to say---I muddle through. image

     

    However, NONE of this conjecture, while fun and entertaining, may out do THIS technology ( http://en.wikipedia.org/wiki/Light_Peak ) in the relatively near future.

     

    ***Somehow, I am not surprised you were in the Navy --- most, if not all, the coolest entrepreneurial kinds of people were in some way, shape, or form militarily involved.  Me, I was just a military brat; close enough to no longer fit in with people, but far enough away to not stand at attention when my SO walks into the room. 

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
Children
No Data
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2026 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube