Free Developement tools from XMOS

A short rummage around the Xcore site lead me to The XCore Open Source Project and the Xcore Github repo, which as an open source advocate, I found very encouraging.  I also noted the contributor's agreement  which goes beyond forking Github repos and allows developers to contribute more directly back to the company sources.  I see this as a sign that open source interest at Xcore is rather more substantial than the "throw code over the wall" kind of openness which is unfortunately common.

Which leads me to a couple of questions:

1. Is the impression I've gained above correct?
2. Are XMOS devices fully documented so that OSHW communities can develop around these devices and create their own complete toolchains to program them?

Question 2 relates to the uncomfortable situation we presently have with FPGA manufacturers, none of which provide complete programming information for their devices so that it has been impossible for the community to develop full toolchains for them.  This has effectively extinguished open source progress in that area.

If the answer is "Yes" to both questions then this is going to be of interest to many.

===

Addendum:  I had asked about unit availability earlier too, but the answer was right here at Farnell UK, 6 different devices all in stock!  Excellent!

Are XMOS devices fully documented so that OSHW communities can develop around these devices and create their own complete toolchains to program them?

We do have public documentation for enough stuff (architecture, ABI etc.) to create a toolchain. Have a look at the following projects for more details:

http://www.xcore.com/projects/gcc-xs1
http://www.xcore.com/projects/binutils-xs1

XMOS Support wrote:

 

Are XMOS devices fully documented so that OSHW communities can develop around these devices and create their own complete toolchains to program them?

 

We do have public documentation for enough stuff (architecture, ABI etc.) to create a toolchain. Have a look at the following projects for more details:

http://www.xcore.com/projects/gcc-xs1
http://www.xcore.com/projects/binutils-xs1

This is certainly right I managed to re-build most of the tool chain  personally to  run on the original RPI ... as you can imagine the performance sucked rather !!

Not  tried it  recently but should work better on the BBB or Odroid.. They did use Eclipse as the IDE and that really needs 1GB or so to be effective

XMOS Support wrote:

 

We do have public documentation for enough stuff (architecture, ABI etc.) to create a toolchain. Have a look at the following projects for more details:

http://www.xcore.com/projects/gcc-xs1
http://www.xcore.com/projects/binutils-xs1

Thank you for replying, XMOS Support.  Would it be possible to define what is not documented please, just in outline or summary?  It's not inherently a problem, but it's important to know where the line is drawn so that it's clear that whatever is not documented does not actually matter for OSHW success.

I'll try to explain why I'm being persistent.  One of our top contributors, John Beetem, has for many years been working on a toolchain to support the "well documented" Cypress PSoC4/5 architecture that embeds FPGA-like elements within an ARM-based host.  The project seemed to have no secrecy hurdles to overcome, but towards the end, very detailed analysis of the register specifications revealed that some of the bit definitions required for configuring PSoC's programmable logic elements had been held back, and so the project fell at the last hurdle.  (This is a second-hand description of events and may not be 100% accurate, but hopefully close enough to explain the danger of partial information.)

This is why I'm trying to understand in advance the extent and limits of XMOS documentation.  There are many kinds of process, fabrication and marketing documentation that are of no interest to OSHW builders and programmers whatsoever, and so their absence would not matter.  But conversely, if information that allows official software to achieve something important (such as configuring logic elements or peripherals or tracing/debugging or any other programmable facility) is omitted from public documentation then it is likely that there will be problems ahead.

I hope this explains what I'm after.  It's really about advance planning, trying to determine whether it is wise or not to begin a multi-year investment of one's time, and it influences purchasing choices.  Knowing the bounds of what is documented allows an informed decision to be made.

Morgaine.

Morgaine Dinova wrote:

 

I'll try to explain why I'm being persistent.  One of our top contributors, John Beetem, has for many years been working on a toolchain to support the "well documented" Cypress PSoC4/5 architecture that embeds FPGA-like elements within an ARM-based host.  The project seemed to have no secrecy hurdles to overcome, but towards the end, very detailed analysis of the register specifications revealed that some of the bit definitions required for configuring PSoC's programmable logic routing elements had been held back, and so the project fell at the last hurdle is examining what can be done.  (This is a second-hand description of events and may not be 100% accurate, but hopefully close enough to explain the danger of partial information.)

Thank you for referencing my work.  Just to clarify for other readers, I'm working on a system for hardware/software co-design where "hardware" in this case is primarily programmable logic.  I would like to make my system capable of going from design entry to bit-stream without needing any proprietary tools.  Unfortunately, most FPGA vendors do not provide the bit-level documentation needed to do this, and if you ask for an explanation they direct you to iamaweasel.com.   Cypress PSoC4/5 is a very promising target, because they do in fact provide bit-level documentation for everything except the routing.  I plan to explore Cypress routing very soon after completing a few other tasks first -- we'll see what I come up with and what I can release.

My general philosophy towards ICs is that vendors should document everything that's needed to use the chip functions for which customers buy the chip.  It's fine to have hidden test modes not used by customers.  I'm willing to live with hidden bit-stream encryption provided that customers can choose not to encrypt and bypass that logic.

Thanks for the improved description, John!

I hope your experience helps XMOS Support to understand my reasons for asking.  It's not an "information wants to be free" kind of thing at all, but a desire that helpful engineering by independent parties not be held back.  A multiplicity of independent tools is highly likely to increase device sales, so there is much to gain for the manufacturer by being fully open in this specific area.

Your defining form of words is extremely well chosen: [highlighting is mine]

John Beetem wrote:

 

My general philosophy towards ICs is that vendors should document everything that's needed to use the chip functions for which customers buy the chip.  It's fine to have hidden test modes not used by customers.  I'm willing to live with hidden bit-stream encryption provided that customers can choose not to encrypt and bypass that logic.

That's a perfect statement of the level of documentation needed, and it is also a very reasonable expectation to have because after all we buy chips so that we can use all their functions. Denying some of those functions to the open source community is not reasonable --- they too are paying full price for the chips after all.  Hopefully XMOS Support will understand that expectation and define the level of documentation using your terms.

One advantage here is that at least one of their Toolchains was openly built around Open source tools.

But as you described so well earlier it doesn't take a lot of "secret source" to ruin your project generally way way down the line after you have put in a whole load of effort.

One advantage here is that at least one of their Toolchains was openly built around Open source tools.

But as you described so well earlier it doesn't take a lot of "secret source" to ruin your project generally way way down the line after you have put in a whole load of effort.

Indeed.  After years of work, that is one surprise that an open source developer really does not need.

Nobody is paying for the work, it's typically a labour of love and freely offered to the world, yet the existence of such additional toolchains is strongly to the commercial benefit of the company in numerous way.

Education is an important issue for me owing to past involvement.  Among other benefits, open tools make a hardware platform more attractive to education through independence from commercial update cycles, they allows bugs discovered by students to be understood and fixed rapidly in-house without delaying courses (often fixed by the students themselves!), and they enable research students to extend the toolchains for specific project aims, a common desire and an easy way of starting graduates off in new directions.  (Highly relevant since parallelism is an important research area.)

Outside of education the benefits of open systems are very widely known of course and in 2013 no longer even need stating.  Perhaps in light of recent highly public events though, it's worth noting that open software can be assigned a far higher degree of trust and security than any closed proprietary toolkit.  When you're designing control systems for a nuclear power plant involving cryptographic security, or you simply want to protect your family or property, that is not a matter to be taken lightly.

Contact seems to have dried up, although of course XMOS Support  might be simply busy, or investigating the question with the intention of giving a well-informed answer.  I hope that's the case.

Meanwhile ...

John Alexander wrote:

 

One advantage here is that at least one of their Toolchains was openly built around Open source tools.

Which toolchain is that?  I've been looking at the Xcore github repos and haven't found any XMOS toolchain sources yet.  The two mentioned toolchain projects (gcc-xs1 and binutils-xs1) appear to be one-person efforts and no files have been posted for either of them.  There are tons of application modules in Github, but they're generally written in XC --- is that compiler maintained as open source?

If not, then since XC has extensions not present in gcc, the point made by John applies to it.  Is all the functionality employed within the XC implementation documented, so that a fully-functional equivalent open source tool (assuming that XC is not) could be written from the documentation?

Hi,

The XMOS toolchains is built around various open source tools such as gdb, LLVM, eclipse, and binutils, and standard intermediate formats such as ELF and dwarf.

Source tarballs for the open source components are on <http://www.xmos.com/products/design-tools-source>.

Hope this helps,

Henk

Thanks for that link Henk, the previous toolchain used Eclipse asthe front end what are they using now??

Sorry haven't installed it yet thus the question!

Sorry but I also see you are using Open OCD which is great, however I do not see support for th XJTAG dongle mentioned in the supported devices column for  OpenOCD.

Has your work been pushed back into the main tree or kept here. It would be nice to use the supplied XJTAG for other stuff if we could ....bummer since 've just got another from Seeed

Henk Muller wrote:

 

The XMOS toolchains is built around various open source tools such as gdb, LLVM, eclipse, and binutils, and standard intermediate formats such as ELF and dwarf.

 

Source tarballs for the open source components are on <http://www.xmos.com/products/design-tools-source>.

That's excellent news Henk, many thanks for the information --- I'll examine them shortly to see what they contain.

Mostly I expect to find what the package names would suggest of course, but the point I'm trying to get answered is the one worded so clearly by John Beetem earlier, namely "vendors should document everything that's needed to use the chip functions for which customers buy the chip" --- the implied question being, "Is this so for XMOS?".

So as an example, if XC is not among the open source tools, then could the full functionality of XC be implemented in an open source tool based on released documentation, or does XC make use of XMOS funtionality or interfaces or protocols or registers etc etc which are not documented?  The question stems from the (hopefully obvious) point that even open source users buy chips to use all their functions.

The question in the context of XC was just an example.  It becomes entirely moot for XC if it turns out that XC is part of an open source package such as XDE, which appears to be another name for xTIMEcomposer which does contain an "xcc" which seems likely to be the XC compiler.  I'll have to check this, but indications seem very hopeful so far just from looking at names.

John's question becomes irrelevant for any tool that is already open source, only applying to those which aren't.

"John's question becomes irrelevant for any tool that is already open source, only applying to those which aren't."

I assume that's John Beetham's question ?

Yep, sorry, name clash, which I thought the context would have kept clear but I guess it didn't.

John Beetem wrote in post #14:

 

My general philosophy towards ICs is that vendors should document everything that's needed to use the chip functions for which customers buy the chip.  It's fine to have hidden test modes not used by customers.  I'm willing to live with hidden bit-stream encryption provided that customers can choose not to encrypt and bypass that logic.

The implied question for any new device being considered as a target by open source developers is whether this level of documentation exists, or whether some closed vendor tools are able to do things beyond the potential of open source because they exercise undocumented features of devices or interfaces, or require secret information such as encryption keys.

It doesn't apply to vendor tools that are already open source since there is usually no need to re-implement them, and even if reimplementation is felt necessary then the open sources are themselves a form of partial documentation.

Hi John (Alexander),

THe IDE is based on eclipse, and there are of course command-line-tools.

Re openOCD I have to disappoint you: we have moved beyond that. We mostly use the XTAG2 debugger which is customisable in software. The good news is that both the hardware design and source code are on-line (https://github.com/xcore/proj_xtag2). As it loads its firmware through a USB-bootloader we can load it with any firmware. The loading process is also on-line on <https://www.xmos.com/en/download/public/Dynamic-code-loader-for-XTAG2.pdf>. This is possibly not for the faint hearted and definitely not part of a gentle introduction to XMOS.

Cheers

Henk

Hi Morgaine

Mostly I expect to find what the package names would suggest of course, but the point I'm trying to get answered is the one worded so clearly by John Beetem earlier, namely "vendors should document everything that's needed to use the chip functions for which customers buy the chip" --- the implied question being, "Is this so for XMOS?".

I fully agree with the spirit of that statement; but there are practical limitations that any company (or individual, or university) will hit. For example, resources are always limited, even in mega companies, and one will first document those parts that the majority of the customers need; or it could be that a company has bought some IP that it is not allowed to release publicly.

For XMOS - we sell silicon and are keen on people to use it in anyway they see fit. The combination of (a) the architecture manual and (b) the system manual should enable anybody to write a compiler, or any piece of assembly code; using any of the datasheets you should be able to write yourself a loader, and configure the chip to the same extent that our tools do. The JTAG debugging interface has (at present) no user-facing documentation, but it is not kept secret, all source code that uses it is on-line in the github repo listed in the previous post.

Cheers,

Henk