Interpreter Project: Metaprogramming

In Crafting Interpreters the many AST Tree Node classes that represent the different kinds of expressions and statements that the Lox language supports are generated with a tool. It generates the actual .java files in the correct package based on data passed into the tool. The classes themselves are very simple data holders, for example:

static class Binary extends Expr {
  Binary(Expr left, Token operator, Expr right) {
    this.left = left;
    this.operator = operator;
    this.right = right;
  }

  final Expr left;
  final Token operator;
  final Expr right;
}

I decided to take that concept a step further and generate the classes dynamically. You can see my code at this point here.

Storing the Data

The YAML file format is ubiquitous in Ruby. It's a no brainer to store the data for generating the classes in a YAML file. What is the data though? I need to generate classes and each class has a different set of attributes and potentially a different base class. Pretty simple, this is what I came up with:

class_name:
  extends: base_class
  attributes:
    name: type

The YAML representation of the Binary java class shown above would be:

binary:
  extends: Ringo::Expr
  attributes:  
    left: Ringo::Expr
    operator: Ringo::Token
    right: Ringo::Expr

The attribute types aren't being used currently. I am leaving the types in place for the time being in case they become useful at some point in the future.

Dynamic Code Generation

I know Metaprogramming is a big thing in Ruby. Everything is an object, even classes and modules, which makes it easy to manipulate and create code at run-time. I haven't spent much time working with Ruby in that way though, so it was a nice learning experience. I read through about half of Metaprogramming Ruby, which is a great book for learning these techniques and really helped me understand the concepts.

Most of it turns out to be very simple. Need to create a new class at runtime? Just klass = Class.new and there you go. klass references your newly created Class. Need it to derive from something other than Object? No problem, klass = Class.new(String).

One neat trick I learned while trying to get the initialize methods to work correctly was the zip method. I couldn't create an argument list for all of the different methods, so it takes a variable argument list *args. How do you assign the correct elements from the argument array to the correct data attribute then? The zip Array method helps out here.

# I get an array of the attribute names:
arg_list = data['attributes'].keys

# That looks like ['left','operator','right']

define_method :initialize do |*args|
  # After the zip the array will look like:
  # [['left', '22'],['operator','+'],['right','12']]
  arg_list.zip(args) do |name, value|
    # Then I can just iterate over the name / value pairs
    # And assign to the instance variables.
    instance_variable_set("@#{name}", value)
  end
end

Moving Forward

Parsing is next on the list. It should be fun and challenging to break up plain text into a well structured syntax tree. I'm hoping the code won't take too long to get working. See you in a week or so.