Language Oriented Programming: The Next Programming Paradigm

Part II. Introduction to Meta Programming System

Creating Languages in MPS

I have explained why we need to easily create new languages. But how can we make it easy? If you turn the question around and apply Language Oriented Programming to itself, you will soon see the answer. This calls for a little self-referential bootstrapping, which can seem tricky, but be patient. Once you understand this, you will 'get' the real power of LOP.

Recall that the idea of LOP is to make it easy to create special domain-specific languages, and those DSLs will make writing our programs easier. But as I've shown, 'programs' in LOP are not restricted to mean the typical 'set-of-instructions programs you are used to. Any unambiguous solution to some problem in some domain is a 'program'. So if you imagine the domain of 'creating new languages', then a 'program' in that domain would actually be a definition of a new language itself, which can be thought of as a solution just like any other solution.

So, applying the idea of LOP, the way to make ‘creating new languages' easy is to create special DSLs dedicated to the domain of ‘creating new languages'. By using these language-building DSLs, we can make it easy to make new languages. Let's look at some of these language-building languages to give you a better idea of how this works. This is an overview; future articles will describe these languages in more detail.

Structure Language

At the bare minimum, we need to define the ‘structure' of a new language. This is how we will be able to write ‘precisely defined' programs. The structure of a language doesn't mean its textual grammar—as I mentioned, there may not even be a textual representation of the program, but only a graph representation.

In most cases, while practicing LOP, you work with two ‘levels' of programming, the meta level and the program level. You define the language in the meta level, and write the program in the program level. When defining the structure of a new language, you would use a language-structure DSL to define your new language, which would reside in the program level.

In MPS, each node in the program level has a ‘type' which is just a link to another node in the meta level. The node in the program level is said to be an ‘instance' of the type. The meta level ‘type' node defines what relationships its instances can have and also what properties they will have. The language for describing this meta level language structure is called simply the Structure Language.

To define a language's abstract syntax with the Structure Language, you should just enumerate all the types in the language. The types simply represent the features, or concepts, that the language supports. Each concept should be defined by its name, the internal properties of its instances, and the relationships (basically links) its instances can have with other nodes (see Figure 3).

figure 3
Figure 3: Definition of the "Method" concept in the Structure Language

There are two kinds of relationships possible. The first kind is an aggregation-like relationship which forms the parent-child tree structure of concept models. The second kind is a non-aggregating, freeform relationship which can link any node to any other node in the system. Relationships have two ends, the source end and the target end. Relationships have roles, and for every role you define the name of the role, the cardinalities of each end, and the type of the target nodes. Cardinalities can be 1, 0..1, 0..n, or 1..n, which lets you restrict how many links can be created for this relationship. The relationship target type can be used to restrict what types of nodes can be linked together.

So, using the new language to write a program would involve creating instances of the concepts in the language, assigning values to the properties of the instances, and linking the nodes in the program together according to the relationships defined by the language concepts. All of this will be supported by powerful editors which you can define for your language.

Editor Language

So, what is the interface for writing and manipulating concept models? We need some sort of editor for our languages. But we don't want a generic editor; experience has shown that generic editors aren't as usable as we want them to be. We want writing models to be fast, so we want specialized editors tailored to our language concepts. In a way, the editor is part of the language, and our goal is to create new languages easily, so creating new editors should also be easy. Essentially, we need a language for creating editors. In MPS, it is called the Editor Language.

When people hear me say that our programs will be stored as graphs and we will have special editors, I'm sure many will think that I'm talking about diagram editors. This is not the case. Even though the programs are graphs, the editors don't have to render as diagrams. In fact, diagram editing is usable in only a small percentage of cases (i.e. when it is appropriate, such as with database tables). In contrast, there is a much better source of inspiration for our Editor Language, and that ironically happens to be text editors.

If you look at a typical program in a text editor, you can imagine that the editor is divided into rectangular cells. Some cells would contain required symbols like keywords, braces, and parentheses, and other cells would contain user-defined symbols like class and method names. Larger cells would be composed of smaller cells, like a method block containing statements, which might also have their own nested blocks. In fact, any well-formed program in any mainstream programming language could be composed into a set of rectangular cells. Well, in the Editor Language, you don't have to imagine these cells, because the editors simply are composed of rectangular cells (see Figure 4).

figure 4
Figure 4: Definition of an editor for the "Method" concept

The usage of cells has some interesting advantages. First, the cells can perfectly mimic, and even out-do, standard text editors while working directly on the program graph instead of text. Second, cells are not limited to text; you could have anything like color choosers, math symbols, charts, vector graphics, or anything else in a cell. In the end, even this cell layout is optional and the programmer could provide a different mechanism. The cell layout is just a useful default.

So the Editor Language helps you define the layout of cells for each concept in the language. You can define which parts are constant, like braces or other decorations, and which parts are variable and need the user to define them. The Editor Language also helps you add powerful features to your own editors, like auto-complete, refactoring, browsing, syntax highlighting, error highlighting, and anything else you can think of. So you can add the power of today's editors, like IntelliJ IDEA, to your own custom languages. This is possible because programs and languages are structured as graphs, and because we have a specialized Editor Language that helps us create powerful editors.

Page
Sergey's photo

Sergey Dmitriev
JetBrains

Sergey Dmitriev is the cofounder and CEO of JetBrains Inc., makers of the IntelliJ IDEA Java IDE .
Sergey's personal website can be found at www.sergeydmitriev.com

Contact Sergey via email: dmitriev (at) jetbrains.com