Domain-specific Languages

Martin Fowler, Rebecca Parsons

Mentioned 8

Martin Fowler's breakthrough practitioner-oriented book on Domain Specific Languages - will do for DSLs what Fowler did for refactoring! * *Fowler's highly anticipated introduction to DSLs: a category-defining book by one of the software world's most influential authors. *Two books in one: a concise narrative that introduces DSLs, and a larger reference that shows how to plan and develop them. *Helps software professionals reduce the cost and complexity of building DSLs - so they can take full advantage of them. Domain Specific Languages (DSLs) offer immense promise for software engineers who need better, faster ways to solve problems of specific types, or in specific areas or industries. DSLs have been around for several years, and have begun to grow in popularity. Now, Martin Fowler - one of the world's most influential software engineering authors - has written the first practitioner-oriented book about them. Fowler's legendary book, Refactoring, made software refactoring a crucial tool for software engineers worldwide; this book will do the same for DSLs. Fowler has designed Domain Specific Languages as two books in one. The first --a narrative designed to be read from 'cover to cover' - offers a concise introduction to DSLs, how they are implemented, and what are useful for. Next, Fowler thoroughly introduces today's most effective techniques for building DSLs. Fowler covers both 'external' and 'internal' DSLs, a well as alternative computational models, code generation, common parser topics, and much more. He provides extensive Java and C# examples throughout, as well as selected Ruby examples for concepts that can best be explained using a dynamic language. Together, both sections enable readers to make wellinformed choices about whether to use a DSL in their work, and which techniques to employ in order to build DSLs more quickly and cost-effectively.

More on Amazon.com

Mentioned in questions and answers.

I did some searches on developing a DSL in visual studio. At the beginning, I found out there is a Visualization and Modeling SDK for VS2010. It has a tool called DSL tool. But it seems that it is only for graphical DSL development.

Then I saw some posts saying "Oslo" is a tool for developing textual DSL, which "was" a Microsoft product - Microsoft no longer supports the tool. http://blogs.msdn.com/b/modelcitizen/archive/2010/09/22/update-on-sql-server-modeling-ctp-repository-modeling-services-quot-quadrant-quot-and-quot-m-quot.aspx

Therefore, I would like to know if I want to develop a textual DSL, what tool is the best? What do you think if I implement a DSL parser making use of F# powerpack with FSLex and FSYacc?

Martin Fowler has a new book about DSL:s. Sadly, it won't discuss much about Microsoft's tooling nor functional languages.

Microsoft no longer support the graphical tool "Quadrant", but MGrammar is still supported and integrated to SQL server, right? MGrammar is the "DSL making language".

Still, I would say that functional languages (read: F#) are the way to go.

This book has a simple example of how to make a DSL with F#: http://www.manning.com/petricek/ and also Google finds many other good references about this topic.

I am new to Antlr, but have used Flex/Bison before. I want to know if what I want to do using Antlr is possible.

I want to parse an PDDL file using Antlr and build up my own representation of the PDDL file's contents in a Java Class that I wrote as the PDDL file is parsed (in the actions for the rules?). After the file is finished parsing I want to return the object representation of the file's contents to the Java program to run other operations on.

So essentially, I want to invoke an Antler produced PDDL parser on a PDDL file from inside a Java program and have it return an object that describes the PDDL file to the main Java program.

Is this possible? I have tried looking at the documentation, but haven't found a good answer.

Thanks very much.

This is certainly possible, since Antlr is designed to generate parsers that then get invoked as part of a larger system (eg, a compiler or a static code analyzer).

Start with Terence Parr's The Definitive Antlr Reference: Building Domain-Specific Languages. He's the Antlr author, and also an unusually clear and jargon-free teacher on language processing.

Martin Fowler's Domain-Specific Languages uses Antlr in a lot of its examples. For instance on page 200 he shows a simple "Hello World" example where a Java program calls Antlr to parse a file of people to greet, and while doing it emits the greetings. Here's where the work gets done (page 206):

class GreetingsLoader. ..
  public void run() {
    try {
      GreetingsLexer lexer = new GreetingsLexer(new ANTLRReaderStream(input));
      GreetingsParser parser = new GreetingsParser(new CommonTokenStream(lexer));
      parser.helper = this;
      parser.script() ;
      if (hasErrors() ) throw new RuntimeException("it all went pear-shaped\n" +
 errorReport() ) ;
    } catch (IOException e) {
      throw new RuntimeException( e) ;
    } catch (RecognitionException e) {
      throw new RuntimeException( e) ;
    }
  }

A third good book is Terence's new one on DSLs Language Implementation Patterns. He describes various ways to use Antlr, as for instance to write an abstract syntax tree generator to put into a compiler.

I'm creating audio and effects in my game on the fly with very basic sound synthesis. Basically, I have some methods that can play a sound given a frequency & amplitude & duration.

For short phrases and melodies, I'd like to come up with a basic notation so that I can easily rewrite or add new melodies into the code (in the end maybe I could read from files, but that is probably overkill).

I'm not sure how to implement this however.

I've started by creating an enum EqualTemperamentTuning which contains all 88 basic piano notes with a MIDI # field and a frequency field. This at least means I can deal in note names and not frequencies.

public enum EqualTemperamentTuning {
    A_0         (1, 27.5),
    A_SHARP_0   (2, 29.1352),
        ...
    C_8         (88, 4186.01);

    private int num;
    private double frequency;

    EqualTemperamentTuning(int num, double frequency){

        this.num = num;
        this.frequency = frequency;
    }

    public double getFrequency(){
        return frequency;
    }

    public double getNum(){
        return num;
    }
}

Then I started creating more objects, first a Note that holds an EqualTemperamentTuning, an amplitude and a length:

public class Note {

    /** Note frequency */
    private double frequency;
    /** Note Amplitude */
    private double amplitude;
    /** Note length */
    private int length;     

    public Note(EqualTemperamentTuning tuning, double amplitude, int length){

        this.frequency = tuning.getFrequency();
        this.amplitude = amplitude;
        this.length = length;   
    }

    public double getFrequency(){
        return frequency;
    }

    public double getAmplitude(){
        return amplitude;
    }

    public int getLength(){
        return length;  
    }
}

Finally to define the melody I want to play, I created a NotePhrase class:

public class NotePhrase {

    /** The arrayList of notes*/
    private Note[] notes;

    public NotePhrase(Note[] notes){

        this.notes = notes;
    }

    public double getFrequency(int counter){

        // for each note in the array
        for (int i = 0; i< notes.length; i++){

            // for each unit of length per note
            for (int j=0; j<notes[i].getLength(); j++){

                counter--;

                // return the frequency at this point
                if (counter <= 0) return notes[i].getFrequency();

            }
        }
        return -1;  
    }
}

Then in my audio generating class I have a loop (with counter) that generates samples from a wave generator. Every time I need a new sample to play, it sets the frequency of the wave according to the NotePhrase.getFrequency(int counter) method above. This should (I haven't actually tested yet!) just play the NotePhrase melody according to the frequency and amplitude (to be added).

Problem is, it doesn't seem very elegant and more specifically it's very difficult to "write" a melody in any kind of legible way. I have to code a whole bunch of new Note objects and then construct a NotePhrase object with an array of them... It's not obvious to me how I would hard code a bunch of these melodies and then easily switch between them later.

Really I would like to create an enum of Melodies or something like that, where I can easily hard code a human legible configuration for each different melody, and then when I want to use them, just pass the enum type to the audio player...

The best I've got is:

private static enum Melody {
    NOKIA_RINGTONE ( new Note(EqualTemperamentTuning.E_5, 0.5, 1), new Note(EqualTemperamentTuning.D_5, 0.5, 1))
    ;

    private Note[] notes = new Note[2];

    Melody (Note note1, Note note2){
        this.notes[0] = note1;
        this.notes[1] = note2;
    }
}

Which I would then load into a NotePhrase object at runtime. This isn't much good because I have to create a new constructor for melodies with different amount of notes. If I do it the other way round and construct the enum with an array of notes, then I'm just "writing" the melody elsewhere and in two parts which seems even more confusing...

So I'm stuck as to how to structure this properly? ie what classes to create and what info they should hold... I want to get this "right" because, I might like to expand the notation in the future to include effects (echo etc) and I've already found with my very little experience, that the right classes, structure and relationships (even names) can make my programs very easy or difficult to understand.

Apologies for the essay, this might not be a very well phrased (ahem) question, but as a java & OOP beginner, any hints would be very welcome!

EDIT**

Thanks for the answers & suggestions, very helpful. Thinking about the answers given in this case triggered me to rethink my general audio implementation, which is pretty shaky right now. Not sure who I should mark as correct though, as I'm really just going to take all recommendations on board and try and go from there.

What comes in mind is implementing a simple internal DSL which will be as close as possible to the notation you use to write down the music notes. In java it could look like

MelodyTrack  track1 = MelodyTrack().create().setTiming(TimeSignature.C)
.eighth(Notes.E5)
.bar().half(Notes.B5).quarter(Notes.A5).quarter(Notes.B5).bar().half(Notes.C6)

there is a good book on DSLs, written by Martin Fowler

This is just an example of how it could look like, I do not insist on one form or another - the basic idea is that your DSL must be readable both by the programmers and domain-experts(musicians). And the way the data is represented in your sequencer should not affect the way you write down the melodies i.e. to make it harder.

P.S. If you are doing something serious, you should try to use an external DSL a.k.a. a midi file or something like that and a sequencing library of some kind, and to store music in the separate resource files of your application.

I have a current need to greatly increase my knowledge around DSLs. Who are the big names in DSLs? What blogs should I be reading?

Martin Fowler has written a book on DSLs (Amazon link). Ayende Rahien has completed a book on writing DSLs in Boo, which also is available online via Manning's early access program. The former is conceptual while the latter is a practical guide.

I'd really like to know what knowledge do I have to pursue in order to create scripting languages from the ground up, probably using C++, maybe python, for those are the languages I know.

I've searched through SO similar questions, but nothing comes close of what I'm asking. I intend to understand language processing, not natural language, but scripts that doesn't have the focus on human readability, only on functionality.

My first idea, without any technical background, was to create a scripting language that assimilates a bunch of logical statements and making use of advanced built-in algorithms like logical reasoning, pattern recognition, neural networks an statistical analysis, in order to provide lots of useful(or not) information.

Sorry my bad english, I learned online, as almost everything I cited here :D and that kinda explains my lack of theorical background.

Thanks in advance.

"From the ground up" is a quite relative term, especially if you consider Python as the implementation language. I think what you are looking for is the implementation of a domain specific language (DSL). Good starting points might be this book or this one. DSLs are a wide topic, so if you provide more details, we might be able to give better tips.

Which Book is best for learning Groovy DSL's? I came across this book : Groovy For DSL's . Apart from this book are there any other books, blogs,site tutorials for learning DSL's using Groovy from scratch?

I didn't find any other book on the subject. I'm waiting for the second edition of Groovy in Action, but it seems DSL part is light

Not related to Groovy but Domain-Specific Languages by Martin Fowler is quite a good book for explaing scope of dsl, examples and patterns.

I heard from someone that DSL is really powerful in some specific fields. So i want to find out if i can put it into my skill sets.

The first problem came out is What is DSL exactly? After doing some search, it seems Groovy supports DSL very well. Then i go and read Groovy's documents and try it out by myself.

And i got the impression that DSL is just some kind of configuration files consisting of texts, XMLs and you use some tools like Groovy to parse it, it magically become some methods or functions you can invoke. What happened?

I read something, but cannot get it straight. Any Help?

Did you read this? Martin Fowler is an authority on the subject and a great writer. I doubt that anyone will improve on the first paragraph. If you still don't get it, give it some time and re-read the article a few times.

I'd recommend looking into JetBrain's MPS

A book might be overwhelming, but there's a relatively new one available.

And i got the impression that DSL is just some kind of configuration files consisting of texts, XMLs and you use some tools like Groovy to parse it, it magically become some methods or functions you can invoke. What happened?

I don't think your impression is entirely accurate. I'd forget about Groovy and parsing and all the implementation details for now. Focus on the problem that DSL is trying to solve.

A DSL designer tries to come up with a pseudo programming language that an expert, who is unfamilar with programming languages like Groovy or Java or C#, would recognize as a simple language describing they way they solve problems.

The DSL uses terms and concepts familiar to any one knowledgable about that domain.

The DSL shields users from the underlying implementation details so they can focus on how to attack their problems.

A DSL is written for the convenience of business users, not developers.

Keep that in mind and the rest is implementation. Eye on the prize....

I'm embarking on writing a language (call me crazy). This will initially be for a test/proof of concept, but hopefully, it will be something I can "use in anger" later on.

I'm looking for a good tutorial that starts from the beginning and then dosen't stop till it gets to the end (well, almosty the end). My objective being to be able to formally specify existing grammars first and then to try writing my own grammar later.

Incidentally, since the target language will be for .Net, does anyone know if there is a parser/generator to produce IL from a grammar?

Note: I know this is a non trivial exercise.

Domain Specific Languages by Martin Fowler

It goes into everything you need (other than the IL specific parser generator).