ACFactory Day 2: ExtendableEnum and XAML Serialization

This is the second part of a series documenting the development of ‘ACFactory’ (ACFabrik in German), an application that generates printable character sheets for the Pen & Paper role playing game Arcane Codex (English page).You might also want to read Day 1: SealedSun goes WPF.

ExtendableEnum

Talents in Arcane Codex have a number of properties that are best described as enumerations. Unfortunately plain old enumerations are very flat. Translating them (when displayed in a user interface) requires you to wrap each and every appearance in the system. This is why I need a richer enumeration type.Enter ExtendableEnum, an abstract base class that handles comparison and parsing of enumeration values. An existing enumeration could even be extended by a plug-in, should my application ever implement a plug-in system.I was, however, confronted with a very annoying problem: type initialisation is lazy. The CLR employs certain “heuristics” to find out when to initialise a type. By default, a type is marked with “beforeFieldInit”, which means that the type is usable before its static fields have been initialised. Of course, static fields are always initialised “just-in-time” when they are accessed. The attribute is not applied when the class contains a static constructor (or class constructor or type initializer). In that case, the type is initialised when one of its members is first accessed.While this is a pretty good strategy for the “normal” use of types, it might be a problem in XAML-based applications since the ExtendableEnum-parser is used before any of the enumeration values is referenced in code, which in turn means that the corresponding enumeration types had no chance to register their enumeration values with the corresponding registry.One Solution I have found is the System.Runtime.CompilerServices.RuntimeHelpers.RunClassConstructor method which, well, runs type initializers. I can only hope that the method (which points right into the heart of the CLR) is smart enough not to initialise a type twice. My approach is not perfect as extensions of enumerations are not necessarily included. For a future plug-in system, some sort of InitializeOnLoad attribute could save the plug-in writers day. But my hack works 100% for all non-extended enumerations and that’s enough for milestone 1.

XAML Serialization

Step 1: Make your objects “expressable” in XAML

The “read”-aspect of XAML serialization is much more important as it is an absolute requirement for milestone 1. The tricky thing with XAML is, that you cannot express circular references for plain CLR objects (no DependencyObject). My object model, however, requires two way relationships in some places. For instance: Talents need access to the attributes of “their” hero in order to compute effective talent levels. Now since there has to be a default constructor, the hero reference will be initialised with null, resulting in an invalid state. There is no way to ensure that your object is initialised correctly, as you don’t know when WPF/XAML is “done” with its manipulations.The only option is to propagate the hero reference down the hero graph once the hero is created, which means that even collections have references to the hero they belong to.

Step 2: Make your objects serialize correctly

Contrary to what people might tell you, XAML Serialization does not come for free. There are severe limitations and not that many customisation options. Here is how I wished I could store my heroes:

<Hero ShortName="Kyle" FullName="Kyle MacDuncan"><Hero.Attributes><Attribute Level="8">Strength</Attribute>...</Hero.Attributes><Hero.Talents><Talent Level="5">Sword</Talent>...</Hero.Talents></Hero>

Automatically converting the hero-less Attribute and Talent to their bound equivalents, HeroAttribute and HeroTalent respectively. Interpreting this is one thing. A bit of Voodoo magic and a couple of virgins (read: TypeConverters and the like) would make this work. But as I said, there is no way to tell the XAML serializer to first convert certain values to more serializable equivalents.So eventually I gave up and implemented  XAML serialization using a pretty nasty hack: All the properties that need processing prior to assignment are loaded off into a xData class (HeroData, TalentData, and so on). This essentially means that I have to implement each non-trivial property at least twice and that extension via plug-ins has become at least twice as difficult. My serialised hero looks like this:

<Hero x:Key="codeHero"xmlns="clr-namespace:ACFabrik.Model"xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation"ShortName="Kyle" FullName="Kyle Mac Duncan"ExperienceTotal="14" ExperienceUsed="12"FameTotal="15" FameUsed="10" Encumberment="0" ><HeroData LocalLibrary="{wpf:StaticResource defaultLibrary}"><HeroData.Attributes><HeroAttribute Level="8" Attribute="Strength" /><HeroAttribute Level="7" Attribute="Constitution" />...</HeroData.Attributes><HeroData.Talents><HeroTalent Level="7">Alchemy</HeroTalent><HeroTalent Level="8">Attention</HeroTalent>...</HeroData.Talents></HeroData></Hero>

The HeroData node defines a new attribute: “LocalLibrary”. It is used to map the talent names (and possibly others) to their definition in an external library. This way multiple heroes can share the same talents. LocalLibrary is only required when heroes are loaded as part of XAML resources, i.e. when you don’t have control over the XamlReader.Load method.The result is a bit more verbose and not that beautiful.Well, at least it will yield good compression ratios (the “<Hero”-prefix…).

Next Steps

On the data side, the next question to answer is how the file formats look exactly. While I now have the basic capability to serialize my objects to streams, I need to come up with concrete formats for heroes and libraries.Also, I need to start thinking about the general UI concept. Printing in WPF works via Visuals, so I literally get previewing for free. I therefore include a very basic UI (preview + print button) in the requirements for milestone 1.

ACFactory Day 1: SealedSun goes WPF

This is the first part of a series documenting the development of ‘ACFactory’ (ACFabrik in German), an application that generates printable character sheets for the Pen & Paper role playing game Arcane Codex (English page).Before I explain why I chose to implement ACFactory in WPF, let me quickly sketch the application’s functionality planned for the first iteration:

  • Read a hero definition from a human-writeable text file
  • Print a sheet that lists all the talents known by a hero

Note that having a graphical UI is explicitly not a requirement for the first iteration. It is not the first character sheet generator I’m writing. The Heldenfabrik (Hero factory, not released) project for instance uses the Prexonite scripting engine to define a DSL used to specify heroes and the same GDI+ based rendering framework (read: bunch-a-classes) as the DSA calendar generator (DSA Kalendergenerator) to generate an almost-A4-sized png. Needless to say, that it was very painful to hard code all the spacings and layout characteristics. I needed something that supported me in designing fixed-page layouts. One solution would have been to write a custom rendering engine that would be tailored to the special needs of fitting content of variable size onto a fixed page. I even went so far as to prototype a possible architecture in F#, only find out that

  1. It is really hard to express complex object models in F# due to the lack of forward declarations.
  2. It is a lot of work to implement automated layout.

So what are the alternatives? What publicly available layout engines exist out there? One obvious answer is XHTML+CSS. XHTML documents are relatively easy to generate (System.Xml.Linq aka LINQ to XML or XLINQ), well understood and can be printed by modern browsers. The problem: XHTML documents are inherently digital and thus optimised for variable sizes. Also the final look depends on the layout engine (read: browser) used to display the document. I don’t even want to talk about the horror that printing from a browser is.Living in a .NET 3.5 environment I could no longer ignore our little newcomer to the graphical user interface world: Windows Presentation Foundation (WPF). We all suspected, that this user interface framework has more to offer than a ridiculously unpronounceable name and fancy 3D graphics. It comes with layout features web designers can only dream about (one word: Grid). So what’s the catch apart from acknowledging that WPF might be cool?A friggin’ steep learning curve. You can learn C# without a book if you’ve already worked with the .NET framework (via, say, VB.NET). You can learn CSS, XML, XPath, XSLT without a book given good online tutorials. But WPF is so complex, that it would take you months to harvest even basic knowledge of WPF from blogs and tutorials all over the net.Windows Presentation Foundation Unleashed (WPF) (Paperback)Since I want to finish the project within 7 days, this was not an option (also, I’m a very impatient person). I needed a book. There were two books recommended all over the internet (Applications = Code + Markup by Charles Petzold) and Windows Presentation Foundation Unleashed by Adam Nathan. Based on the customer reviews I decided to buy the latter (the fact that this one was printed in colour made the decision much easier :-) ).So here I am, roughly in the middle of the book on page 355 at the beginning of chapter 11 about 2D graphics. One thing I noticed was the alarming number of pitfalls or things-to-remember when coding WPF interfaces. Although the majority of the framework builds on predictable and consistent patterns, there are odd edges here and there such as having to use {Binding RelativeSource={RelativeSource …}} for relative sources as opposed to the uniform {Binding Source={RelativeSource …}. Overall I have the feeling that there will be a lot of reinventing-the-wheel involved as WPF is not as complete as it could (should?) be. Sortable ListView out of the box, anyone?But wait! Didn’t I forget something? Right: Like XHTML, WPF is tailored towards flexible layouts in containers of variable size. Doesn’t this rule WPF out as the layout engine of choice? Nope: WPF was designed to be resolution independent, operating on “virtual” pixels the size of 1/96th of an inch (or device pixels at DPI=96). I can therefore calculate the size of an A4 page at 96 DPI, which is roughly 793.701 pixels times 1122.52 pixels or 3.77953 pixels per millimetre (Beautiful number isn’t it? :-( ).

Next Steps

Before I start designing the character sheet, I want to have my Business Objects (the hero + his/her talents) to be up and running. I need a format that is human-writeable and can be loaded by my application. I’m currently looking into the object serialization aspect of XAML, which I already successfully used to define a serializable library of talents available to heroes.XAML (aka System.Windows.Markup) knows how to serialize objects that

  • Have a public default constructor (no arguments)
  • Expose their representation in writeable properties

It’s not that adhering to those rules is especially difficult but it is extremely tedious as immutable objects are essentially ruled out. I like to build my objects (classes) like fortresses, using the type system, readonly modifiers and immutablity to reduce the number of ways an object can enter an invalid state. Designing a business model that serializes to XAML therefor means opening all doors to malicious (and/or stupid) users (developers that use my code). I practically live in the library-writer-world and having to write class that expose their representation the way WPF likes it, makes me twitch.But seeing how elegant XAML maps XML elements to the familiar .NET objects really makes up for the mental pain I am about to go through. You’ll hear from me.Continue reading on ACFactory Day 2: ExtendableEnum and XAML Serialization

Operator overloading for Prexonite, more or less

Prexonite October 2008

While Prexonite Script does not formally provide any means of object-oriented programming (not just consuming objects but actually defining new ones) there are mechanisms in Prexonite (the runtime) that try to make up for this.

What Prexonite includes so far

  • The Structure data type
    • groups values and functions together
    • provides "natural" access via dot-notation

    function create_complex(_re, _im)
    {   
        _re ??= 0.0;
        _im ??= 0.0;
        var z = new Structure;
       
        z.\("re") = _re;
        z.\("im") = _im;
       
        z.\\("norm") = (this) =>
            sqrt(this.re^2 + this.im^2);
        z.\\("add") = (this,other) =>
            create_complex(this.re + other.re,this.im + other.im);
        z.\\("ToString") = (this) =>
            this.re + " " + this.im + "i";
       
        return z;
    }

  • The ExtendableObject base class
    • Makes it possible for Prexonite code to "extend" .NET objects, should the developer want to allow this.
  • The struct function (in psr/struct.pxs )
    • automates construction of Structure objects via reflection

    build does require("psr\\struct.pxs");

    function create_complex(_re, _im)
    {   
        _re ??= 0.0;
        _im ??= 0.0;
       
        function re(this, new_re)
        {
            if(new_re is not Null)
                _re = new_re;
            return _re;   
        }
       
        function im(this, new_im)
        {
            if(new_im is not Null)
                _im = new_im;
            return _im;   
        }
       
        function norm = sqrt(_re^2 + _im^2);
       
        function add(this, other) =
            create_complex(re + other.re,im + other.im);
       
        function ToString =
            re + " " + im + "i";
       
        return struct;
    }

New in this release1

  • auto-property, proxy-property, property support via psr/prop.pxs
    build does require("psr\\struct.pxs","psr\\prop.pxs");

    function create_car(a_color)
    {     
        _re ??= 0.0;
        _im ??= 0.0;
       
        function re = struct_prop(_re);
       
        function im = struct_prop(_im);
       
        function norm = sqrt(re^2 + im^2);
       
        function add(this, other) =
            create_complex(re + other.re,im + other.im);
       
        function ToString =
            re + " " + im + "i";
       
        return struct;
    }

  • operator overloading via (+), (==) etc.
    //...
    function (+) this other =
        create_complex(re + other.re,im + other.im);

    function (-.) =
        create_complex(-re, -im);
       
    function (-) this other = this + (-other);
       
    //...

1psr/prop.pxs is in the repository since August.Let's discuss those two extensions one by one…

Properties

First of all, there still is no language support for structures and properties. As of right now, they are implemented via compile time transformations. These two transformations can be enabled via importing the Prexonite Standard Repository scripts psr/struct.pxs and psr/prop.pxs. While the former has been around for some time now, the latter has only recently been implemented in managed code.psr/prop.pxs defines the special function prop which is expanded into get and set code depending on the way it is used. The easiest mode of operation implements a property with an anonymous backing field (defined in the surrounding scope, i.e. as a global variable in a top-level function or a shared variable in the outer function) The second mode mimics a simple get-set proxy where the expression passed as an argument is used as the backing field. This works for arbitrary expressions as long as they can be the target of an assignment (i.e. they are get-set expressions). The third mode finally is a replication of C# properties taking separate lambda expressions for its get and set implementations.struct_prop works the same way but ignores the additional this parameter for structure methods.

Operator overloading

When resolving operator calls, generic objects (as well as structures) now also try to call special instance methods after failing to find a corresponding static operator overload. This makes the implementation of operator overloads for structures and ExtendableObjects (An abstract class that makes an object extendable in the same way a structure can be extended) possible. In order to avoid strange operator names, I have introduced operator ids, special literals (on the lexical level) that represent operator names. There is nothing special about these literals, they can be used everywhere an id can be used. You could for instance define a local variable with the name (+).

SealedSun.GamePanel June08

[ Download SealedSun.Gamepanel June08]SealedSun.GamePanel is a high level wrapper around the managed LcdInterface by http://www.cabhair-cainte.com/ismiselemeas/ that features a Winforms-like controls system as well as simplified event routing for the four soft buttons of the G15.The library is not very mature and only the most commonly used features are implemented. Everyone with basic knowledge of System.Drawing should be able to implement custom functionality via user controls.

Quick Start

1. Create a new LcdScreen

The LcdScreen represents an entry in GamePanel manager and can be selected by the user via the application switch button on the Keyboard.

using(var screen = new LcdScreen())// ...}

2. Open the LcdScreen

To notify the GamePanel manager of the new screen, call the Open method.

screen.Open("My LCD");

3. Create a new LcdPage

An LCD page is like an empty canvas. It is the container for all your UI elements.

var page = new LcdPage(screen);

4. Add controls to the LcdPage

There are currently only two built-in general purpose controls: LcdLabel and LcdImage. In order for your controls to show up, you need to add them to the page object. You can also nest controls by adding them to other controls. Always keep in mind, that the positioning is relative to the parent control and that controls cannot exceed the bounds of their container.

var lTitle = new LcdLabel();lTitle.Text = "My LCD";lTitle.Location = new Point(10,10);page.Controls.Add(lTitle);

5. Configure soft buttons

The page object provides access to the four soft buttons via its Button0,1,2,3 members. Those special controls provide 'button pressed' events and are always located immediately above the respective buttons. Their Text and Image properties can be used to label them. If you need better control over the labelling, you can add child controls to the soft buttons like to any other control.

page.Button0.Pressed += (sender,e) =&gt; {Console.WriteLine("0 Pressed!");};

6. Select active page

In order for a page to be displayed on a screen, it must be assigned to its CurrentPage property. Only pages that are currently active will fire events.

screen.CurrentPage = page;

7. Enter message loop

Like a windows forms application, an LCD application too has a message loop. Similar to Application.Run, screen.Run will use the current thread to process messages from the GamePanel manager.

From here on...

If you want to get the users attention, you can change the pages screen priority to Alert for a short amount of time. A convenient way to do so is the TemporaryAlert method. This of course only works, if the user allows high priority screens.

page.TemporaryAlert(500);

If you want to quit the message loop, either close the screen or call Screen.ExitMessageLoop.

License and Disclaimer

Except for the components mentioned in the ExternalResources.txt, I hereby release everything the zip archive under the terms of the Creative Commons Attribution-Share Alike 2.5 Switzerland license.The software was written over the course of a few days and did therefor not undergo much testing. It is most likely that you will encounter bugs and unexpected behaviour. You use the library at your own risk.Creative Commons LicenseSealedSun.GamePanel by Christian Klauser is licensed under a Creative Commons Attribution-Share Alike 2.5 Switzerland License.

Prexonite May 2008 Update

Downloads

Update

There have not been any significant changes since the introduction of the CIL compiler into the Prexonite, yet the current version comes with a number of performance optimizations regarding the generated CIL byte code.The majority of the built-in commands and types now use the ICilCompilerAware interface, which is used by the CIL compiler to let commands and types emit highly customized code. Calling println with no arguments for instance, results in a static call to void System::Console.WriteLine() directly in the compiled method.Similarly, type expressions in CIL functions are no longer implemented via type expression parsing but by directly referencing the corresponding singleton PType objects.But the most important improvement is the possibility to statically link Prexonite function calls in CIL compiled methods, which makes yet another hashtable lookup redundant at the cost of additional memory: A dynamically generated class has static fields for each and every function used by the compiled application. This can be a problem if you plan to re-compile your CIL-implementations, as dynamic type, unlike dynamic functions, cannot be garbage collected by design. It is, however, possible to disable the generation of such a class by passing false to CompileToCil.And on a side node: The often used library function struct has been implemented as a compiler hook for improved performance. By resolving the members at compile time one does not only save run time, but also removes the need for dynamic lookups, which in turn enables the use of CIL compilation for struct-functions. This is especially helpful for immutable structs.

CIL compilation hints and their effects

*Update 2008/02/20* I have just merged the CIL compiler branch into the trunk. The CompileToCil command is now officially part of Prexonite:Prexonite source code (trunk)In the last article, I presented the Prexonite CIL compiler and the huge performance improvements it comes with. Unfortunately, the compiled code has to be dynamically typed as the CIL compiler does not perform any data flow analysis and can therefore not possibly infer the correct types. It does not even create its own representation of the byte code program.However, to say Prexonite Script (the language) is strictly dynamically typed is actually wrong, as the Prexonite compiler emits code for which the types and even the method overloads are known at compile time. It's just that the virtual machine does not provide a way to take advantage of this knowledge.One such example is the foreach loop, a construct that consists of

  • An expression (the list)
  • A block of statements
  • A left-hand value (the element)

and gets transformed into

var enum = $list$.GetEnumerator~IEnumerator;while(enum.MoveNext()){$element$ = enum.Current;$block$;}

This pseudo code represents what is emitted by the Prexonite compiler for foreach AST nodes. It is clear that enum has to be at least of type IEnumerator in all cases. This information could enable the CIL compiler to statically type the variable enum, turning two late-bound calls (MoveNext and Current) into virtual calls.

CIL compilation hints

CIL compilation hints are basically a reverse mapping from byte code to AST nodes, reduced to the minimal amount of information required by the CIL compiler to emit optimized code. It is not that the whole AST is now encoded in the Meta tables of functions. Only nodes, for which the CIL compiler could generate better code, emit CIL hints.One example is the foreach node, which emits the name of the enumerator variable and the addresses of the late-bound calls to be optimized. The CIL compiler decodes this information and performs the necessary steps. The enumerator variable for instance will be of type IEnumerator<PValue> and won't be initialized ahead of time.

Impact on performance

The two main paradigms to interact with sequences in Prexonite are the combination of coroutines (sequence operators like where and map) and the use of foreach loops. While coroutines have the advantage of compose ability and deferred execution, foreach loops are usually faster.Again, I used micro benchmarks to demonstrate the impact on performance. For practical reasons the number of iterations depends on the size of the set to iterate over in the inner loop. N = 200'000 makes the basis. With sets of 10 and 100 elements, N is reduced to 20'000 and 2'000 respectively.Iterations over a set (Measurements)What you are seeing here are performance improvements of 950 to 3'400%, but keep in mind that those are very specialized micro benchmarks and that unless your program exclusively consists of mindless foreach loops, you will not likely experience such speed-ups.Nonetheless, iteration over lists is a very important aspect of many of the programs I have written in Prexonite Script.

Prexonite CIL Functions

Save the "What the f..." for later and just look at the two snippets below.

ldloc.1ldc.i4.5addstloc.1

Listing 1: a = a + 5 in CIL assembler

ldloci  1ldc.int 5addstloci  1

Listing 2: a = a + 5 in Prexonite assembler

On the left you see four CIL assembler op codes, while the other snippet represents the exact same program, just written in Prexonite byte code assembler. The fact that the two programs look so similar is no coincidence as the Prexonite virtual machine was actually modelled after the CIL’s execution model. This exact similarity can be exploited to make Prexonite a lot faster.

A Prexonite to CIL compiler

Now before you get too excited, Prexonite Script still is what they call a “Dynamic Language” and a lot of its features are implemented in the underlying Prexonite virtual machine instead of the language compiler. Also, Prexonite byte code is not statically typed, which makes a straight translation to CIL impossible without very sophisticated data flow analysis and complete type inference. As I am not familiar with either of these topics, I decided to keep the Prexonite functions untyped. This is where the PValue class comes into play. It encapsulates a dynamically typed piece of data and provides many methods to interact with the contained data via late binding.

In all cases, an implementation of a Prexonite function in CIL must show the exact same behaviour as the original, interpreted implementation. Functions that interact with Prexonite stack frames cannot be compiled to CIL as they are no longer executed on the virtual machine’s stack but the CLR’s instead. Therefore, CIL implementations must be able to exist alongside interpreted implementations and that as transparently as possible. Also, since the Prexonite virtual machine allows for code generation and manipulation at runtime, CIL implementations must be replaceable. This unfortunately also means that function calls inside CIL implementations cannot be statically linked as the target function might change the implementation strategy (interpreted, CIL) every moment.

How it’s done

Since the Prexonite to CIL compiler operates on Prexonite byte code, it would not make much sense to use the C# or VB CodeDOM and the corresponding compiler. Instead System.Reflection.Emit provides the necessary API. Since implementations must be replaceable, dynamic types are not an option and the so called lightweight functions are used.

The compiler is designed to operate at runtime, invoked by the running program itself. This is, because it analyses the whole application to identify functions that are not compatible with compilation to CIL. Such functions are marked with the Meta entry volatile.

The compilation process itself is actually quite straight forward. First the function is analysed in order to determine the number of temporary variables required, to build up a symbol table and to identify shared (via closures) and non-shared variables. Then the common function header is emitted including the creation of PVariable objects for shared variables and the initialisation of non-shared variables with PType.Null.
Then, the variables representing arguments are initialised with either PType.Null or the value supplied in the arguments array and finally the special variable args is set to a list of those same arguments if required by the function.

What follows is a huge loop that iterates over every instruction in the functions code and passes it into a giant switch statement, which translates every Prexonite byte code instruction into a series of CIL op codes.

Therefore, the CIL implementation of the program in Listing 2 will look like in the pseudo CIL in Listing 3.

As you can see, an untyped implementation of this simple program expands into quite some code. Notice that due to the absence of a rotation op code, the implementation requires temporary variables to insert the local stack context in the call to Addition.

ldloc var1ldc.i4.5box int32call IntPType PType::get_Int()newobj instance void PValue::.ctor(object, PType)stloc temp1ldloc sctxldloc temp1call instance class PValue PValue::Addition(StackContext, PValue)stloc var1

Listing 3: Actual CIL implementation of the program in Listing 2
Note: I have shortened the fully qualified type names for better readability.

Is it worth the effort?

As with all optimization techniques, we must ask ourselves whether the effort for implementing it is worth the gain in performance (be it memory or speed). At this point, let me just throw the results of an amateurish micro benchmark at you.

CIl_micro_benchmark

One can clearly see that CIL implementations are superior. They perform the same tasks in 60% (empty_loop) to 30% (rec_echo x 100) of the time required by the interpreted versions. Since the CIL compiler performs many of the Meta data lookups required for the creation of a stack frame at compile time, function calls to CIL implementations are much faster. Keep in mind though that only interpreted functions can take advantage of tail calls. To prevent an overflow of the managed stack, you should implement infinite recursive loops in interpreted functions.

Overall, you could say that compilation to CIL will result in a free performance improvement of over 65 percent in most cases.

function rec_echo(n) =if(n == 0)else1 + rec_echo(n-1);
function rec_echo_direct(n,r) =if(n == 0)relserec_echo(n-1,(r??0)+1);

A functional touch

The last days, I've been working on two things: The reorganization of built-in commands and the improvement of the "Functional Experience".Why do commands need reordering? Because it gets difficult to find the right file among over 40 commands.Why the sudden increase in numbers? I added proxies for System.Math methods for both easy and fast access to mathematical functions such as Sqrt and Sin, but also Pi.Additionally, the most important coroutines from the Prexonite Standard Repository for list processing have been implemented in managed code, again for performance reasons. Map, Where, Limit, Skip and friends now inject managed coroutines into the stack.The commands are now organized in the namespaces Core, List, Math and Text. The latter currently contains the fixed layout functions SetCenter, SetLeft and SetRight, which fill a given string with some character sequence until it has a certain length and is aligned correctly.Now what the hell do you mean by "Functional Experience"?I haven't told anyone but the Prexonite VM is absolutely terrible when it comes to recursion. Unfortunately, recursion happens to be one of the key elements in functional programming and, as you might have noticed, Prexonite Script comes with a lot of syntactic sugar that makes it look like a functional programming language.Ok, lambda expressions and closures are "true" functional features but the lack of a sophisticated type system makes it almost impossible to reason about a program in the way functional compilers do. Nonetheless, I added two features with the last commit, that make PXS a tiny little bit more functional.

First of all: Tail Call Optimization
Yes, the thing that helps with recursion.
function fac(n,r) =
    if(n == 1)
        r
    else
        fac(n-1,n*r);

I benchmarked this function three times, with different tail call optimization strategies. The difference is huge. See for yourself (10'000 computations of 16!):Comparison of different tail call optimization strategies.Two strategies are employed: An implementation of tail call optimization for directly recursive functions inside the compiler, that turns recursive calls into direct iterations (jumps to the beginning of the function with different arguments). What I call "virtual machine optimization" is a special tail call instruction that removes the current stack frame after having called the function or closure.Now apparently the virtual machine "optimization" is not particularly fast but uses far less memory than the normal invocation.Prexonite will never be able to recognize indirect recursion due to the lack of control flow analysis. This, however, does not mean that return statements inside conditions or calls in tail position are not recognized. I'm not sure if Prexonite will ever handle simple recursive return expressions like the normal definition of the factorial:
function fac n =
    if (n==1)
        1
    else
        n*fac(n-1);

Also in the repository is an experimental and partial implementation of the famous call-with-current-continuation from Scheme. In PXS it is known as callcc.I must admit that I don't really know much about call/cc and how it works, especially regarding the stack. Creating a callable object from the current state of a function invocation is no problem. I just don't understand some of the scheme samples, I've been looking at (terribly difficult to read...)The following snippet stores a continuation of the function two in the global variable plusone. Invoking this continuation with, say, 6 returns 7 as the name suggests.
var plusone;

function two =
    1 + call\cc(->one);
       
function one(continuation)
{
    plusone = continuation;
    return 1;
}

The Philosophy Behind: The Prexonite Type System

This is the second article in the "Philosophy Behind"-series, picking up a specialty of one of my projects and explaining how it came to be made. Last time I wrote about the "auto dereferencing" concept in Prexonite Script.In today's article I will explain the reasons behind the design of the Prexonite type system.Prexonite faces the same problem as other implementations of late-bound languages on the .NET platform: How to map the CTS to the languages type system.Prexonite_TypeSystemI think the basic types Int32, Double, Boolean and String are more suited for a statically typed environment, so my type system must allow me to provide wrappers around third-party classes/structs.Wrapping and unwrapping objects must be as transparent as possible. Return values from base class library methods have to be wrapped in their Prexonite equivalent.At the same time, it is not practical to write a custom wrapper for every possible C# or VB.NET library, so there must be some sort of universal wrapper for CLR objects. With users of Prexonite being able to write their own wrappers, it must be possible to have multiple wrappers for the same CTS type. Also, some wrappers might handle more than one type.The solution for Prexonite is the abstract class PType and some concrete subclasses, including the universal ObjectPType, which does all the late binding. Since Prexonite Script performs type checks at runtime, type information has to be associated with every data object, which is just what the class PValue does.What might surprise you, is the fact that Null is considered a type. Every null reference automatically has type Null. Unlike the sturdy null references in C#, instances of Prexonite Null are completely functional objects. They react to operators, can be converted to basic values (Int, String,...) and even provide a ToString method. However, Null does have a special position in the Prexonite type system: it is not possible to write and use your own null reference wrapper.

Prexonite standard repository finally released.

I just checked the collection helper functions I call Prexonite standard repository (psr) into SVN. As I am too lazy to create a full release, I will just supply you with a trac generated zip file.

Following is a short documentation (or rather an overview) of psr:

The Prexonite Standard Library is a collection of scripts that help in day-to-day hacking with Prexonite Script. This page shortly outlines the contents of each of the currently available files.

debug.pxs

Dependencies
none

The script enables special treating of the debug command using compiler hooks for increased performance. For each call to the debug command, it checks whether the function requests debugging (through the debugging MetaKey). Unless that is the case, the call will be removed. if-Blocks using debug as their condition will be evaluated at compile time in respect to the debugging key.

It is possible to use the debug command without including this script, in that case, however, your scripts will also contain calls to debug when not being debugged.

The actual functionality of this script has been moved to managed code inside the Prexonite.dll for performance reasons in #18. CompilerHooks have to be used with care. While the loss in compiler performance is barely noticeable with just one user defined CompilerHook, many of them can really slow the translation down. The managed implementation uses a shared CompilerHook to further save time, should Prexonite.dll ever include additional CompilerHooks

[Read more →]