Interpreter Project: Metaprogramming

Published on May 3, 2018

In Crafting Interpreters the many AST Tree Node classes that represent the different kinds of expressions and statements that the Lox language supports are generated with a tool. It generates the actual .java files in the correct package based on data passed into the tool. The classes themselves are very simple data holders, for example:

static class Binary extends Expr {
  Binary(Expr left, Token operator, Expr right) {
    this.left = left;
    this.operator = operator;
    this.right = right;
  }

  final Expr left;
  final Token operator;
  final Expr right;
}

I decided to take that concept a step further and generate the classes dynamically. You can see my code at this point here.

Storing the Data

The YAML file format is ubiquitous in Ruby. It's a no brainer to store the data for generating the classes in a YAML file. What is the data though? I need to generate classes and each class has a different set of attributes and potentially a different base class. Pretty simple, this is what I came up with:

class_name:
  extends: base_class
  attributes:
    name: type

The YAML representation of the Binary java class shown above would be:

binary:
  extends: Ringo::Expr
  attributes:  
    left: Ringo::Expr
    operator: Ringo::Token
    right: Ringo::Expr

The attribute types aren't being used currently. I am leaving the types in place for the time being in case they become useful at some point in the future.

Dynamic Code Generation

I know Metaprogramming is a big thing in Ruby. Everything is an object, even classes and modules, which makes it easy to manipulate and create code at run-time. I haven't spent much time working with Ruby in that way though, so it was a nice learning experience. I read through about half of Metaprogramming Ruby, which is a great book for learning these techniques and really helped me understand the concepts.

Most of it turns out to be very simple. Need to create a new class at runtime? Just klass = Class.new and there you go. klass references your newly created Class. Need it to derive from something other than Object? No problem, klass = Class.new(String).

One neat trick I learned while trying to get the initialize methods to work correctly was the zip method. I couldn't create an argument list for all of the different methods, so it takes a variable argument list *args. How do you assign the correct elements from the argument array to the correct data attribute then? The zip Array method helps out here.

# I get an array of the attribute names:
arg_list = data['attributes'].keys

# That looks like ['left','operator','right']

define_method :initialize do |*args|
  # After the zip the array will look like:
  # [['left', '22'],['operator','+'],['right','12']]
  arg_list.zip(args) do |name, value|
    # Then I can just iterate over the name / value pairs
    # And assign to the instance variables.
    instance_variable_set("@#{name}", value)
  end
end

Moving Forward

Parsing is next on the list. It should be fun and challenging to break up plain text into a well structured syntax tree. I'm hoping the code won't take too long to get working. See you in a week or so.