OCCAM to XC - Source to Source translator

[jonathan]

I am considering writing a set of program transformations from OCCAM to XC at the level of a basic unoptimised parse-tree and then generating equivalent XC code. Given the highly formalised structure of OCCAM, I suspect most of this is really quite achievable and it should be fairly simply to translate basic OCCAM programs - should have that up and running in a few days.

More advanced OCCAM language features may need to be unpacked a bit - for example into assembly language routines, bypassing XC.

This is just an idea at present. Has anyone here had a look at doing this/have any experience/advice to offer?

Thanks.

[andrew]

You could look into writing a OCCAM front end for LLVM. That way you would get a lot of the hard work done for you. There is also a cpp backend on it already which you might find useful.

[octal]

I agree with Andrew, an LLVM fontend would be the best option. But, is the XMOS LLVM backend fully working?
Does actual XC version use LLVM for backend ?

[Folknology]

I would be great to see Occam on XS1

regards
Al

[richard]

I agree with Andrew, an LLVM fontend would be the best option. But, is the XMOS LLVM backend fully working? Does actual XC version use LLVM for backend ?

The C compiler we ship uses LLVM as a backend, so at least for the set of things you can do in C it's production quality. The XC compiler currently doesn't use LLVM for the backend but we've recently added XCore specific intrinsics to LLVM to support for I/O operations and events. These changes are publicly available from the upstream LLVM repository (see http://llvm.org).

The following post on the Occam pi mailing list discusses possible ways to get Occam running on the XCore:

http://occam-pi.org/list-archives/occam ... 01670.html

[richard]

This is just an idea at present. Has anyone here had a look at doing this/have any experience/advice to offer?

You might want to take a look at Sire (https://github.com/xcore/tool_sire) since this is a case where XC is used as the target language of a compiler.

[octal]

This is just an idea at present. Has anyone here had a look at doing this/have any experience/advice to offer?

You might want to take a look at Sire (https://github.com/xcore/tool_sire) since this is a case where XC is used as the target language of a compiler.

:shock:
I checked the source codes (in Python) yesturday (very very quickly) and I tought that the Parser/Lexer used to use Python's version of YACC/LEX, and the code generator was generating native ASM code directly.

I need to check it again.

Anyway doing the translation to XC is not that good I think. There will be so many problem like slicing arrays that need a lot of XC code for support, and the second MAIN problem I can see is the fact that you'll lose all the source level debug (at OCCAM source level), as the DWARF generated files will map to the XC sources and not OCCAM sources.

[jonathan]

Octal - I think sire can do both from having another quick look. We're probably best asking Jamie himself, but I think being able to target XC is very handy for something like sire. I have a few clues why, but don't want to speculate.

Although I understand there are likely to be a few complexities with source to source translation/compilation, I'm going to see where I get to, and then look at an LLVM front-end to see how much additional work it would be (allowing for any progress XMOS may have made in the meantime).

One of the potential stumbling blocks is things such as parallel replicators that use more than the available number of hardware threads. Of course, the transputer had no such limitation. I haven't checked how XMOS deals with things such as this (I assume, as yet, that they haven't in a compiler), but a source to source translation preserves the program structure whereas an attempt to compile to the architecture would require clever tricks - that are best done properly and once only by XMOS.

The other thing is that I'd really like to see the XMOS XC-based LLVM work, before investing heavily in getting my head around the "infrastructure" of LLVM.

[Jamie]

Octal - I think sire can do both from having another quick look. We're probably best asking Jamie himself, but I think being able to target XC is very handy for something like sire. I have a few clues why, but don't want to speculate.

An XC conversion is definitely a quick way to get things working, allowing you to directly use constructs such as loops and conditional statements. Things get trickier when you want to implement features not supported by XC, for example, sire allows array reference parameters which means they have to be implemented manually (i.e. with assembly inlines for loads and stores). This is still probably better though than writing a full-blown compiler.

I think the most difficult part (and has caused quite a few problems with my sire implementation) is the issue of threading. XC has it's own threading model with accompanying runtime which is pretty specific to the architecture and offers little flexibility. To get occam up and running, you would need a proper virtual threading scheme allowing an arbitrary number of threads (bounded by memory) to be created on a core. Edit (to clarify): A full implementation of such a scheme doesn't exist yet to my knowledge; paul's vThreads project does offer some of this functionality, but this is a possible direction for the sire work, although it's not clear yet whether this is the best option.

Assuming you have a decent virtual threading scheme you need it to be tightly integrated into the compilation process which I think would be difficult with an XC-based approach. It would be far more appropriate to use LLVM here.

[daveg]

One of the potential stumbling blocks is things such as parallel replicators that use more than the available number of hardware threads. Of course, the transputer had no such limitation. I haven't checked how XMOS deals with things such as this (I assume, as yet, that they haven't in a compiler), but a source to source translation preserves the program structure whereas an attempt to compile to the architecture would require clever tricks - that are best done properly and once only by XMOS.

The other thing is that I'd really like to see the XMOS XC-based LLVM work, before investing heavily in getting my head around the "infrastructure" of LLVM.

I don't know a great deal about how Occam did/does things, but the number of threads used in an XC program is checked by the mapper (xmap). The compiler just generates the requisite thread requests and doesn't care about how many you are asking for, leaving it up to the mapper to throw an error if more than 8 are requested simultaneously on any particular core. The spec of XC only allows for threads to be created by a par statement and doesn't allow for dynamic thread creation/destruction outside of this construct.

If Occam wants an unbounded number of threads and dynamic creation/destruction of threads then you'd need to add an additional layer of software before you could even use XC as a target. Perhaps a library for "virtual" threads?


Note that the Xcore LLVM backend contains intrinsics for the thread (and channel, port, timer etc) creation/destruction/sync, so you could easily target a language which has more dynamic constructs than XC at LLVM using these. For example, it wouldn't be terribly difficult to add use of these to C via these intrinsics and have a hardware threading library for C or C++.


Edit: If you use a source-to-source translation it will be difficult (or perhaps impossible) to control things such as the ABI or structure layout to match what the original language expects.