Extensible Languages for Flexible and Principled Domain Abstraction

Most programming languages are designed for general-purpose software development in
a one-size-fits-all fashion: They provide the same set of language features and constructs
for all possible applications programmers ever may want to develop. As with shoes, the
one-size-fits-all solution grants a good fit to few applications only.
The trend toward domain-specific languages, model-driven development, and languageoriented programming counters general-purpose languages by promoting the use of
domain abstractions that facilitate domain-specific language features and constructs
tailored to certain application domains. In particular, domain abstraction avoids the need
for encoding domain concepts with general-purpose language features and thus allows
programmers to program at the same abstraction level as they think.
Unfortunately, current approaches to domain abstraction cannot deliver on the promises
of domain abstraction. On the one hand, approaches that target internal domain-specific
languages lack flexibility regarding the syntax, static checking, and tool support of domain
abstractions, which limits the level of actually achieved domain abstraction. ... mehrOn the other
hand, approaches that target external domain-specific languages lack important principles,
such as modular reasoning and composition of domain abstractions, which inhibits the
applicability of these approaches in the development of larger software systems. In this
thesis, we pursue a novel approach that unifies the advantages of internal and external
domain-specific languages to support flexible and principled domain abstraction.
We propose library-based extensible programming languages as a basis for domain
abstraction. In an extensible language, domain abstraction can be realized by extending
the language with domain-specific syntax, static analysis, and tool support. This enables
domain abstractions as flexible as external domain-specific languages. To ensure the compliance with important software-development principles, we organize language extensions
as libraries and use simple import statements to activate extensions. This facilitates modular reasoning (by inspecting import statements), supports the composition of domain
abstractions (by importing multiple extensions), and allows uniform self-application of
language extensions in the development of further extensions (by importing extensions
in an extension definition). A library-based organization of extensions enables domain
abstractions as principled as internal domain-specific languages.
We designed and implemented SugarJ, a library-based extensible programming language
on top of Java. SugarJ libraries can declare and export extensions of SugarJ’s syntax,
static analysis, and editor support. Thereby, a syntactic extension consists of an extended
syntax and a desugaring transformation from the extended syntax into SugarJ base
syntax, an analysis extension matches on part of the current file’s abstract syntax tree
iii
and produces a list of errors, and an editor extension declares editor services such as
coloring or code completion for certain language constructs. SugarJ extensions are fully
self-applicable: An extended syntax can desugar into the declaration of another extensions,
an extended analysis can check the declaration of an extension, and an extended editor
can assist developers in writing extensions. To process a source file with extensions, the
SugarJ compiler and IDE inspect the imported libraries to determine active extensions.
The compiler and IDE adapt the parser, code generator, analyzer, and editor of the
source file according to the active extensions.
In this thesis, we do not only describe the design and implementation of SugarJ, but also
report on extensions of the original design. In particular, we designed and implemented a
generalization of the SugarJ compiler that supports alternative base languages besides
Java. Using this generalization, we developed the library-based extensible programming
languages SugarHaskell, SugarProlog, and SugarFomega. Furthermore, we developed an
extension of SugarJ that supports polymorphic domain abstraction and ensures communication integrity. Polymorphic domain abstraction enables programmers to provide
multiple desugarings for the same domain-specific syntax. This increases the flexibility of
SugarJ and supports scenarios known from model-driven development. Communication
integrity specifies that components of a software system may communicate over explicit
channels only. This is interesting in the context of code generation where it effectively prohibits the generation of implicit module dependencies. We augmented SugarJ’s
principles by enforcing communication integrity.
On the basis of SugarJ and numerous case studies, we argue that flexible and principled
domain abstraction constitutes a scalable programming model for the development of
complex software systems.