Wednesday, December 19, 2007

Updates soon

Dear all,
Apologies for my lack of updates.
Things have been going very well with the project, in fact I completed my 3rd year project and received the award for the best project in 2007, which was nice.
I have continued to develop the research surrounding Lucian and have recently submitted a paper to Mathematics in Computer Science detailing this research. I will shortly be uploading the Lucian compiler to the web, please look at my website for shortly appearing details.

Monday, January 22, 2007

Lucian progress & Guido (working title)

It's been a while since I have blogged about my project but I *have* actually been working on my project quite a bit! I managed to do a good chunk of work over the Christmas break and have really developed the Lucian language so that it properly handles nesting now, and all kinds of primitive operations and variables. I have also, this weekend, added functions (see below for an example of a program that performs a running (2*i + 2*(i+1))*fibonacci[i] using functions in Lucian). The demo also demonstrates two kinds of nesting, where and let in. It also shows how great dataflow is, see the recursive power function in one line :) Whoohoo!

fun power x = 1 fby power*x

fun fib =
f = 1 fby f + g
g = 1 fby f

fun powsWithFib =
x = power(2)
y = next power(2)
z = fib()

print powsWithFib()

I also had a little bit of a diversion from my primary objective and started playing with a graphic development for writing Lucian code. I've currently named it "Guido" (a common Italian name throughout the ages) a play on "GUI Do", I'll probably change the name, but that's not important!

I decided that as I was writing the Lucian translator in Haskell and having a lot of fun with Haskell I'd have a crack at writing some kind of graphical tool in Haskell. I chose the wxHaskell extension which uses the wxWidgets packages as it seemed like a well established graphics library that uses the native OS widgets of the target platform. I however had an epic battle when I discovered that the latest wxHaskell, 0.9.4 (released in May 2005), completely fails to compile with the latest wxWidgets 2.8.0 (released in December 2006). So I had a battle taking the wxHaskell 0.9.4 code (written in C++), and writing compatibility hacks and code to work with the latest wxWidgets. If anyone is interested I uploaded my modififed code and wrote a short explanation of how to coax the two to compile together over at my site
A reasonably complete list of the changes I made can be found at

Anyway, once I had all this sorted I spent an afternoon playing with making a little 'flowchart' style functionality tool, examples of how it looks can be seen here:-
Fibonnaci 'code'.

Natural numbers 'code'.

So generally things are going well. I am now working on bringing conditionals into the language, and then I think I will re-write a few bits to make things tidyer and then head into writing Lucian-J the Lucian -> Java translator, then I will formalize the semantics for Lucian and then I should be nearing completion of deliverable 1! :)

Friday, December 01, 2006

Progress Report

I managed to write most of my progress report tonight (which is great news). I'd already made some rough notes on my tablet about what I wanted to write, so that helped a bit.

While doing my progress report I, once again, got to pimp my diagrams. Here is a diagram describing the transformation process that occurs when Lucian is passed an input file, and translates it to an output file. (

Click for the full view

Tuesday, November 28, 2006

Rogues and Revolution

Well for some reason I haven't made a project blog post in over 3 weeks. I guess I have been really busy, quite a few nights of only 4-5 hours sleep. But nevermind!

I am feeling 1000 times better this week after sorting some stuff out, and project work has been steaming ahead since the weekend (not to say that I hadn't been doing any before of course!).

Lucian(R) is going really well, its exploded from a basic parser, into a much fuller parser and translater since my last post, and it also now outputs Ruby code that is syntactically correct. Okay so the code output doesn't appear to do much yet, but I'm confident that its a simple issue. Its a great feeling when pass Lucian a file with a couple of lines of Lucian data flow code and it outputs about 150 lines of Ruby code.
So we are getting there, hazzarrrr!!!

Code fragment of the night: ++ "\n" ++
I seem to keep writing that, making the output code prettier.
Anyhoo, should probably rest for a bit now. Has been a really fun and rewarding coding session though!

(p.s. The new Incubus album is really great, check it out!)

Thursday, November 02, 2006

Work on syntax & semantics for Lucian

Over the past couple of days I have been playing with syntax ideas in my head, and have finally started toying with these on paper.

I have produced an informal document outlining a possible syntactical system for Lucian. I'm sure that it will go through many iterations, revisions and transformations in the next couple of weeks.

The document can be found here in PDF format.

Sunday, October 29, 2006


Following a productive week of reading, and an extremely useful meeting with my supervisor on Thursday I have been able to get a lot more direction about where I am going with project and how the dataflow extension is going to work.

My supervisor has managed to (temporarily) prize me away from lazy streams for the moment, which is good. I need to get the extension working eagerly first, and that is going to be the main objective.

I have put together a short document detailing the nature of the dataflow extension I will be developing, and also a revision of my project timetable.

Progress and Timetable Revision 1

The current name for my extension is Lucian (a play on Lucid of course, and in reference to Lucian of Samosata and the artist Lucian Freud).
Here is the description of Lucian taken from the above document:-

The nature of the dataflow extension has been further solidified over this week leading to the creation of this document. The current name I am using to label this extension is Lucian.

    The dataflow extension will take the form of a pre-processor that interprets a dataflow language, L, and outputs source code in a target object orientated language, A. The dataflow language will consist of an abstract syntax for expressing dataflow over objects, and will allow the programmer to reference objects that exist within local source code in the language A.

    For example the programmer writes a class definition for an object X, in the language A, along with supplementary code in the dataflow language L, which describes some dataflow involving X. The Lucian pre-processor then generates all the dataflow code in the OOP language A.

    The Lucian pre-processor will also have the ability to parse source code written in the language A that has embedded dataflow code appropriately delimetered within.

    Hence the Lucian language is a meta-programming language, and the pre-processor acts as the generator to the target language A.

    Initially I will be developing Lucian-R, an implementation of Lucian that outputs source code in the Ruby programming language. Due to planned modularity within the pre-processor the ability to output to any other OOP language, e.g. Java, Smalltalk should be fairly straight forward.

Tuesday, October 24, 2006

Specification, lack of sleep and crushing my mind with lazy concurrent streams

Completed and handed in the specification document for my project last week. The final document can be found here.

The last week has been extremely busy, and so has the weekend, resulting in tangible work being low, although I have managed to press on with reading the Lucid book as well as spent several hours trying to fathom how to produce an efficient system of lazy concurrent streams.

I've made a few attempts at implementing lazy streams with my Ruby Dataflow Extension, but keep getting stumped at the last minute. In my head it seems easy, but when I sit down at the computer to write code I find myself knee deep in partially applied functions, deadlocks and traps. Extending Ruby to allow currying of functions was a definite help.
Here is the lovely code that lets me hack currying and partial application into Ruby:-

# Code based on "Whys" blog post on functional extensions in Ruby.
class PartialFun < Proc
def call(*args)by
if args.size < arity.abs and args.size != -arity - 1{|*a| call(*(args+a)) }

alias_method :[], :call

# CurryFun constructor wrapper

def CurryFun &block &block

# Example function
def curry_foo
CurryFun{|x, y, z| x+(y*z)}

One concept in my head is explainable via the examples now laid out.
Lets take the example:-
x = x fby 1
y = next x

This is all fine and dandy in Lucid, where y will be outputting 1 continually on its stream. Lucid is lazy. In my RDE (Ruby Dataflow Extension), dataflow programming occurs via concurrent nodes on top of threads, not on an eduction model (which I presume Lucid works on), and this concurrent system is eager.
Hence the previous example fails as x is nothing, so 'x fby 1' is left trying to read from x, and 'next x' also sees nothing.

My thought is that if the node 'fby' tries to read from x and does not receive anything, then it outputs on its buffer a closed function that attempts to read from buffer x.
The node then continues as normal and reads 1, and outputs that continually to its output buffer.
The next node reads the "next" item on the output buffer of x, hence skipping over the function on the buffer and reading 1.

However, lets say that we don't have this following program:-
a = some_slow_running_node
x = a fby 1
y = next x
z = x

Taking the model of operation as before, fby may try to read from the output buffer of a, but as a is a slow running node, it doesn't read anything immediately, so flags there is no result yet, or writes out a function. This doesn't matter for y as y only requires next x. However we now have z, which replicates all that is coming from x. If x outputs a function, z could try and apply the function. Lets say that fby didn't read anything from a immediately, so wrote a function onto its output buffer. A function whose operation is to read from a. z would read this function, and could apply it to see if we have a value for a yet. Could this work in a more complex situation? I'm not sure yet.
For example the "howfar" program presented in Section 3 of Wadge's Lucid book:-

howfar = if x eq 0 then 0 else 1 + next howfar fi;

This program calculates how many items there are on the input stream before a 0 is encountered.
E.g. If the input stream to howfar is:-
1, 6, 4, 0, 9, 2, 1, 4, 3, 0, 6...
The output would be:-
3, 2, 1, 0, 5, 4, 3, 2, 1, 0, 1...

With the previous model of passing functions onto streams the expression 1 + next howfar could be rolled into a partially applied function, whose opeartion is 1 + reading the next value of the howfar buffer. More formal reasoning or tests are needed to ascertain if this will be a viable solution to producing concurrent lazy streams.

This does leave me with a nasty taste in my mouth though, it feels like I'm destroying the type system for the streams, as now not only can a stream be of type a, it must also be able to hold functions. Perhaps two streams masquerading as one would be better, one stream presenting the strongly typed item stream, and another subversive counterpart, holding any functions partially applied or not.

I have thought about a request system where results are 'pulled' through the system, with nodes making requests for computation to other nodes, but this destroys the whole joy of having a concurrent data flow network in which some nodes could be operating much faster and could do a lot of useful work while other nodes are taking longer.

The thinking continues, along side getting familiar with Lucid. My objectives for this week are to be researching into GLU and Gypsy, I'm hoping that I can do this at the weekend after much reading of the Lucid book this week.