Language Oriented Programming: The Next Programming Paradigm

Collection Language

The next most important language we need is a language for working with collections. The need for collection support is ubiquitous. Every major mainstream language has some sort of support for collections. For example, in Java you have java.util, in C++ you have STL. Everybody needs collections. If different DSLs had their own collection support, there would be a Babylon of different collection languages, each incompatible with each other. This is why MPS must provide a single Collection Language which everyone uses.

In many mainstream languages, collections are not language features but class libraries. A good example is Java’s java.util package. The support is technically there, but it is so inconvenient, messy, and error prone. Yuck! Most Java code today is littered with lines and lines of redundant, repetitive code for handling collections. Figure 6 shows an example of how a Collection Language beats the tar out of a class library. The example is an algorithm for finding the convex hull of a given set of points. More details about the Collection Language will be forthcoming in future articles.

figure 6
Figure 6: Convex hull algorithm using the Collection Language

User Interface Language

The User Interface Language is the next most important DSL for our platform. Interestingly, the Editor Language I mentioned previously could conceivably be used for providing user interfaces, but a full-fledged language for graphical user interfaces would be more flexible. The benefits of such a language would be enormous. Java Swing code is a perfect example of a class library wanting to be a DSL. The features are there, but are easy to misuse [3], and Swing code is a complete mess. Many environments today include GUI builders to simplify user-interface creation. The User Interface Language will take that mission to a higher plane. I will discuss this language in more detail in future articles.

Getting Started with MPS

I can already hear some of the skeptical responses to LOP: “Sounds great, but our project is already underway and switching to LOP at this point isn’t practical,” or “Sounds great, but it’s too risky to start a real-life project with an untested method like LOP,” or “Sounds great, but when will it be ready for prime time? OOP took 20 years to become mainstream.”

The good news is that you won’t have to dive head-first into the unknown; you can dip your toe in and see if the water is nice first. You can try just a little bit of LOP on your project to see if it provides a practical advantage, and then try a bit more if you like it. Here are two possible applications of LOP which you will be able to try in the near future with MPS.

Using MPS on Java Applications

There is already a prototype plugin for IntelliJ IDEA which will allow you to include MPS concept models in your project. The models will automatically be translated into Java source code in the background as you edit them. So, you will be able to write part of your Java applications using MPS, as much or as little as you want. This means that you get all the power of MPS, such as the ability to create and use specialized DSLs, to make whatever language extensions you want, as well as to use customizable editors with code completion, error highlighting, refactoring, etc. The plugin will be tightly integrated with IDEA, allowing you to embed Java code in your MPS models, navigate to embedded or generated Java code, and even perform concept-level debugging similar to the JSP debugging support already available in IDEA. More integration features are planned, as well. This will be an important new tool available to Java developers using IDEA.

Configuring and Scripting Your Applications

There's a pattern I've seen many times. An application starts off needing some form of configuration, be it a simple options file, or a more complete deployment descriptor file. Eventually, configurations become more complex, and the application ends up needing a scripting language. For simple configuration files, XML is popular. For scripting languages, you can create your own, or borrow a general-purpose one like VBScript, Python/Jython, Tcl, Javascript, or maybe Lisp. Each of these solutions has at least some of the standard drawbacks of mainstream programming: Long time to implement, steep learning curve, hard to extend, poor environment support, etc.

Alternatively, you could create your own configuration/scripting language with MPS. Users of your application would then have an easy-to-use, intelligent editor for writing their scripts, including syntax highlighting, error highlighting, code completion, navigation, etc. It will take a fraction of the time to create and integrate the language into your application. You will be able to redistribute the MPS runtime for use with this kind of application.

Conclusion

The ideas underlying LOP and MPS are not new, and have actually been around for more than 20 years [1]. The term Language Oriented Programming itself has been around for at least 10 years [2]. What is new is that these ideas have silently saturated the software development community, and their time has finally come. With this article, I hope to provide a seed around which these ideas can crystallize into new discussions, opinions, critiques, experiments, research, and real-life projects.

And so, I invite you to take part in this new paradigm in whatever way you can. Add a comment below, or send me an email at mps_article@jetbrains.com. Find out more about MPS at http://www.jetbrains.com/mps and watch for updates. Take a fresh look at websites, magazines, blogs, and books from the perspective of LOP, and think about how much easier things could be. Think about your own projects and how often what you're doing is actually designing and using little specialized languages cobbled together with classes and methods. What do you think about that? I want to know.

I have already seen first-hand how Language Oriented Programming can drastically improve software development, as I have been using the concept of LOP to develop MPS itself. MPS is currently not ready for the real world, but it is getting there. There is also no documentation yet, except for this article. I will publish more articles soon, which will explore MPS in more depth. Also, I plan to make MPS available in a more-usable form in coming months, so keep your ears open.

There are other projects out there that follow similar approaches, notably from Intentional Software, and Xactium.

So have fun exploring, and let me know what you find.

Acknowledgements

I would like to thank Rob Harwood for his help in editing this article. I would also like to thank the following people for their reviews, comments, and suggestions: Igor Alshannikov, Florian Hehlen, Jack Herrington, Guillaume Laforge, Vaclav Pech, Thomas Singer, Dmitry Skavish, David Stennett, and Timur Zambalayev.

References

Articles:

[1] Donald E. Knuth. Literate programming. The Computer Journal, 27, 97-111, May 1984.

[2] M. Ward. Language Oriented Programming. Software - Concepts and Tools, 15, 147-161  1994, http://www.dur.ac.uk/martin.ward/martin/papers/middle-out-t.pdf

Intentional Programming articles

Charles Simonyi. The Death of Computer Languages, The Birth of Intentional Programming. 1995. ftp://ftp.research.microsoft.com/pub/tr/tr-95-52.doc also ftp://ftp.research.microsoft.com/pub/tr/tr-95-52.ps

John Brockman. Intentional Programming: A Talk With Charles Simonyi. Edge. 2000. http://www.edge.org/digerati/simonyi/simonyi_p1.html

Microsoft Research. Intentional Programming. http://www.cse.unsw.edu.au/~cs3141/ip.asf (video)

Charles Simonyi. Intentional Programming: Asymptotic Fun? http://www.hpcc.gov/iwg/sdp/vanderbilt/position_papers/simonyi.pdf

Books:

Krzysztof Czarnecki and Ulrich W. Eisenecker. Generative Programming: Methods, Tools and Applications. Addison-Wesley, 2000. ISBN: 0201309777.

Jack Herrington. Code Generation in Action. Manning, 2003. ISBN: 1930110979. http://www.codegeneration.net/cgia/

Xactium. Applied Metamodelling: A Foundation for Language Driven Development. 2004. http://albini.xactium.com/content/index.php?option=com_remository&Itemid=28

Other Resources on the Web:

[3] Matt Quail. Totally Gridbag. http://madbean.com/blog/2004/17/

Jack Herrington. Code Generation Network. http://www.codegeneration.net/

Intentional Programming interviews

Sergey Dmitriev. http://codegeneration.net/tiki-read_article.php?articleId=60

Charles Symonyi. http://codegeneration.net/tiki-read_article.php?articleId=61

Krzystof Czarnecki. http://codegeneration.net/tiki-read_article.php?articleId=64

Andy Evans. http://codegeneration.net/tiki-read_article.php?articleId=68

Have your own opinion? Discuss this article »

Page
Sergey's photo

Sergey Dmitriev
JetBrains

Sergey Dmitriev is the cofounder and CEO of JetBrains Inc., makers of the IntelliJ IDEA Java IDE .
Sergey's personal website can be found at www.sergeydmitriev.com

Contact Sergey via email: dmitriev (at) jetbrains.com