Tuesday, September 10, 2013

Annotation Semantics Changing for Generics

Annotations are a useful feature that essentially allow you to program the Crack compiler at compile time.  They provide the functionality of macros in C plus a whole lot more - you can actually use them to implement your own domain specific languages, if you like.

Annotations are stored in a "compile time namespace." This is separate from the normal namespace that functions, variables and classes live in, but it follows the same resolution rules: annotations are scoped to the lexical scope in which they are defined.

Until now, annotations used in generics were no different from annotations in any other context.  When the generic was instantiated with new parameters, it was essentially replayed into the same compile time context (including the same compile-time namespace).

Unfortunately, this approach doesn't work with caching.  When a generic is cached, the original compile context is gone.  We have to restore parts of that context (notably, the normal namespace) but the contents of the compile time namespace cannot be persisted or restored: it can contain arbitrary objects created by the annotation system, and, in fact, it routinely contains pointers to primitive functions.

The only way to restore the compile namespace of a generic is to replay the original source file it was defined in.  This would be contrary to the purpose of caching, and it would also require us to create a dummy environment so as not to actually regenerate existing code and representational datastructures.

Rather than try to go down this path and further delay the 1.0 release, I've decided to impose some limitations on the way that annotations can be used with generics.  This was a painful decision: I don't like having non-uniform semantics in the language.  Annotations should work the same for generics as for any other code in a module, but something had to give and after discussing it with the team I feel this is the best compromise.

As of this morning's check-in, generics now preserve only imported annotations.  So for example, the following code will no longer compile:

@import crack.ann define;
@define my_func() { void f() {} }
class A[T] {  @my_func }

It could be rewritten like this:

@import crack.ann define;
class A[T] {
  @define my_func() { void f() {} }
  @my_func
}

Note that the symbols defined with @import can be reused -- it's possible for us to replay imports safely.  But macros defined with @define must be defined (or redefined) within the scope of the generic itself.