Lisp and DSLs

[Didier Verna]

Content preview: Dear friends, I'm starting to write a chapter for an upcoming
book on domain specific languages. The chapter is called (tentatively): Extensible
languages -- blurring the distinction between DSLs and GPLs [...]

Content analysis details: (-101.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/526>


Dear friends,

I'm starting to write a chapter for an upcoming book on domain specific
languages. The chapter is called (tentatively):

Extensible languages -- blurring the distinction between DSLs and GPLs

GPL meaning General Purpose Language in this context ;-)


My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.

Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.


Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.


Thank you very much in advance!

[Yakov Zaytsev]

LOOP macro

Post by Didier Verna
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!
--
Resistance is futile. You will be jazzimilated.
Scientific site: http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[Ryan Davis]

Not sure if I'd call it a "pearl", but here's one data point. Hell, I'm
not even sure if this counts as a DSL or not, most of what I know of
DSLs comes from blog posts and ruby fanatics.

I had a legacy electronic voting system (IIS/VBscript/SQL Server) and
wrote some lisp to configure elections. The lisp code output the T-SQL
needed to configure the election, and then the VBScript was driven from
that data.

Here's an abbreviated example:

(with-election ("3/29/2011 8:00PM" "3/31/2011 7:00PM" "Law")
(with-position ("President" :types '("Law Undergrad"))
(make-candidates "name 1" "name 2" "name 3"))
(make-referendum "Amendment 1" "<strong>Thing To Change</strong>
<p>HTML body for the voting UI</p>"))

The :types indicated who was eligible to vote for the position, and
there were some other options to those macros. The with-* forms are
macros to setup the context, and the make-* functions generate the
appropriate SQL strings and write them to a stream (usually a file).

The macros in the sample do a few things:

1. with-election: sets a dynamic variable in lisp for the stream all
the SQL strings are written to in make-* functions, allowed easy
testing of make-* functions via the REPL
2. with-position: sets a variable in the T-SQL script used by INSERT
statements produced by make-candidates. This actually
macroexpands to imperative code: (progn (make-position ...)
(make-candidates ...)). The macro usage is mostly to get
indentation and visually group the relationships

So, not sure if I'd call this a pearl and it's arguably a DSL, but it's
sure a lot easier for me to use than direct T-SQL or the ridiculous
VBScript web interface.

Other things I've used that might fall into the DSL category:

* cl-who http://weitz.de/cl-who/
* cl-interpol http://weitz.de/cl-interpol/
* clsql's SQL-READER syntax (http://clsql.b9.com/manual/ref-syntax.html)

HTH,

Ryan Davis
Acceleration.net
Director of Programming Services
2831 NW 41st street, suite B
Gainesville, FL 32606

Office: 352-335-6500 x 124
Fax: [phone]

Post by Didier Verna
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!

[Peter Seibel]

I think the chapter on binary data types in PCL is a reasonable
example of a certain kind of DSL. Is this whole book about DSLs? I'll
be interested to see it. And glad they're getting at least one Lisper.

-Peter

 Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!
--
Resistance is futile. You will be jazzimilated.
Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[Didier Verna]

Content preview: Peter Seibel <peter-***@public.gmane.org> wrote: > Is this whole
book about DSLs? I'll be interested to see it. And glad > they're getting
at least one Lisper. Yes. I'll let you know when the project is finalized.
[...]

Content analysis details: (-101.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/530>

Is this whole book about DSLs? I'll be interested to see it. And glad
they're getting at least one Lisper.

Yes. I'll let you know when the project is finalized.

[Alexander Repenning]

We have been working on what we call domain oriented languages for some time ranging from the first visual programming language for the MIT programmable brick (later Mindstorms) http://www.cs.colorado.edu/~ralex/papers/PDF/VL95-LEGOsheets.pdf

to AgentCubes (a 3D environment for end-users): http://www.cs.colorado.edu/~ralex/papers/PDF/AgentCubes_JVLC_article_inpress.pdf

Both of these programming environment are built in Common Lisp, are based on the concept of rules and have domain oriented interfaces to turn visual languages (via Lisp macros) into Lisp code.

Not sure this is what you are looking for but many people (teacher, students, ...) are using these kinds of domain oriented languages to build games and simulations: http://www.9news.com/news/local/article/202987/222/Teachers-play-video-games-for-science-

best, Alex

Post by Didier Verna
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!
--
Resistance is futile. You will be jazzimilated.
Scientific site: http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

Prof. Alexander Repenning

University of Colorado
Computer Science Department
Boulder, CO 80309-430

vCard: http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf

[Paul Tarvydas]

Content preview: It seems to me that most people don't understand the rudiments
of CL's ability for creating DSL's and live in the land of Blub. If you're
going to discuss creation of DSLs, it might be an idea to first relate the
aspects of CL using compiler terminology. [...]

Content analysis details: (0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[98.139.212.173 listed in list.dnswl.org]
0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/532>

It seems to me that most people don't understand the rudiments of CL's ability for creating DSL's and live in the land of Blub.

If you're going to discuss creation of DSLs, it might be an idea to first relate the aspects of CL using compiler terminology.

Here are some random thoughts (not all are accurate, but represent an attempt to translate CL concepts into Blub):

- CL is a language that has Lex and Yacc built into it. You can change the syntax of CL to suit your problem.

- Lisp s-expressions are concrete parse trees.

- The CL "reader" is a scanner. You don't need to build a scanner, since CL already provides you with one.

- CL symbols are "tokens".

- The CL scanner can read programs from a number of sources - not just files - e.g. strings, keyboard, streams, etc.

- Physicists commonly create DSL's. They define a notation with which to describe a domain, then use the notation to solve problems in the domain.

- Few other languages can change their own syntax. Examples of self-modifying syntax can be found in term-rewriting systems, such as the language TXL.

- CL macros are parsers.

- CL macros are code emitters.

- CL macros are source-to-source program transformers.

- CL backquote/comma/at-sign notation is used to splice and modify concrete parse trees into other shapes.

- The CL compiler compiles s-expressions into machine code.

- If one creates s-expressions, one can use the CL compiler to compile them into machine code at runtime.

- The CL compiler is a JIT compiler.

- The CL repl interactively compiles s-expressions to machine code, then executes them, then prints the results of the execution.

- The CL language defines a rich library of operators for modifying concrete parse trees.

- To a CL'er, infix notation is just unnecessary syntactic sugar. Many novice CL'ers try to create macros that understand infix notation early on in their careers, but as they become more familiar with the power of CL, they drop the syntatic sugar and use parse trees directly.

- The CL printer contains a notation for displaying structure sharing. This facilitates debugging of optimizers which transform trees into graphs.

- To write a DSL in CL, one first writes a macro that parses s-expressions. Then, the macro is extended with backquotes to emit new s-expressions. The emitted s-expressions are automatically fed to the CL compiler and compiled to machine code.

- Unfortunately, the C language uses the name "macro" for a facility that is far less powerful than that of CL. In C, the macro facility can only parse a syntax that looks like simple symbols and function calls and a special #if syntax that is not even part of the C language definition. In CL, the macro facility can parse any s-expression and utilitize any CL function during code emission.

- With C macros, the programmer does not have the option to change the syntax of the macro language. With CL macros, the programmer has full control of the macro syntax.

- CL has the ability to link/load modules at runtime, "on the fly".

- CL invented the concept of JIT (is this factually true?).

- Forward references in CL are automatically handled by the runtime linker. The linker fixes up references on the fly to improve runtime efficiency (I know that I first saw this facility in UTAH Lisp in 1977).

- The CL scanner, compiler and linker sit in memory all (most) of the time, hence, CL is an all-in-one bundled DSL/compiler toolkit. Some CL's give you the ability to remove the compiler when creating a .exe, if you no longer need it.

- The builtin CL macroexpand function gives the programmer the ability to debug his/her DSL compiler by viewing the emitted code (much like CPP's command line option that leaves the expanded code in a text file).

- CL's rich suite of syntactic and compiler tools came about due to early AI research into automatic code generation.

- CL lambdas (closures) are thunks.

- CL closures are trampolines.

- CL closures and Scheme's call-with-continuation are the ultimate goto's. Any control flow can be modelled and compiled with these facilities.

- CL closures and Scheme's call-with-continuation can be used to create compilers using the concepts of denotational semantics.

- Cl closures are (kind of) like Smalltalk's "blocks".

- CL closures are another example of the rich language-building facilities of CL, not commonly found in most languages.

- CL closures can be used to create anonymous functions, similarly to anonymous classes used in Java.

- The facilities of CL make it possible to model and compile just about any kind of runtime semantics, including very non-linear ones, e.g. PAIP prolog.

- Prolog provides an simple way to build parsers, if efficiency is not of utmost concern. A number of prologs are available to CL programmers, so one can have the best of both worlds. [I use paip prolog to parse diagrammatic syntax.]

hth

pt

[Marco Antoniotti]

Content preview: Paul gave an excellent overview. I would also describe the
ability of CL of "re-plumbing" itself (shameless plug: the DEFINER library
:) ) and, of course, as a standard example from ages ago, Prolog-like languages
and the slew of logic languages build on top of CL. [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/533>

Paul gave an excellent overview.

I would also describe the ability of CL of "re-plumbing" itself (shameless plug: the DEFINER library :) ) and, of course, as a standard example from ages ago, Prolog-like languages and the slew of logic languages build on top of CL.

cheers

MA

Post by Paul Tarvydas
It seems to me that most people don't understand the rudiments of CL's ability for creating DSL's and live in the land of Blub.
If you're going to discuss creation of DSLs, it might be an idea to first relate the aspects of CL using compiler terminology.
- CL is a language that has Lex and Yacc built into it. You can change the syntax of CL to suit your problem.
- Lisp s-expressions are concrete parse trees.
- The CL "reader" is a scanner. You don't need to build a scanner, since CL already provides you with one.
- CL symbols are "tokens".
- The CL scanner can read programs from a number of sources - not just files - e.g. strings, keyboard, streams, etc.
- Physicists commonly create DSL's. They define a notation with which to describe a domain, then use the notation to solve problems in the domain.
- Few other languages can change their own syntax. Examples of self-modifying syntax can be found in term-rewriting systems, such as the language TXL.
- CL macros are parsers.
- CL macros are code emitters.
- CL macros are source-to-source program transformers.
- CL backquote/comma/at-sign notation is used to splice and modify concrete parse trees into other shapes.
- The CL compiler compiles s-expressions into machine code.
- If one creates s-expressions, one can use the CL compiler to compile them into machine code at runtime.
- The CL compiler is a JIT compiler.
- The CL repl interactively compiles s-expressions to machine code, then executes them, then prints the results of the execution.
- The CL language defines a rich library of operators for modifying concrete parse trees.
- To a CL'er, infix notation is just unnecessary syntactic sugar. Many novice CL'ers try to create macros that understand infix notation early on in their careers, but as they become more familiar with the power of CL, they drop the syntatic sugar and use parse trees directly.
- The CL printer contains a notation for displaying structure sharing. This facilitates debugging of optimizers which transform trees into graphs.
- To write a DSL in CL, one first writes a macro that parses s-expressions. Then, the macro is extended with backquotes to emit new s-expressions. The emitted s-expressions are automatically fed to the CL compiler and compiled to machine code.
- Unfortunately, the C language uses the name "macro" for a facility that is far less powerful than that of CL. In C, the macro facility can only parse a syntax that looks like simple symbols and function calls and a special #if syntax that is not even part of the C language definition. In CL, the macro facility can parse any s-expression and utilitize any CL function during code emission.
- With C macros, the programmer does not have the option to change the syntax of the macro language. With CL macros, the programmer has full control of the macro syntax.
- CL has the ability to link/load modules at runtime, "on the fly".
- CL invented the concept of JIT (is this factually true?).
- Forward references in CL are automatically handled by the runtime linker. The linker fixes up references on the fly to improve runtime efficiency (I know that I first saw this facility in UTAH Lisp in 1977).
- The CL scanner, compiler and linker sit in memory all (most) of the time, hence, CL is an all-in-one bundled DSL/compiler toolkit. Some CL's give you the ability to remove the compiler when creating a .exe, if you no longer need it.
- The builtin CL macroexpand function gives the programmer the ability to debug his/her DSL compiler by viewing the emitted code (much like CPP's command line option that leaves the expanded code in a text file).
- CL's rich suite of syntactic and compiler tools came about due to early AI research into automatic code generation.
- CL lambdas (closures) are thunks.
- CL closures are trampolines.
- CL closures and Scheme's call-with-continuation are the ultimate goto's. Any control flow can be modelled and compiled with these facilities.
- CL closures and Scheme's call-with-continuation can be used to create compilers using the concepts of denotational semantics.
- Cl closures are (kind of) like Smalltalk's "blocks".
- CL closures are another example of the rich language-building facilities of CL, not commonly found in most languages.
- CL closures can be used to create anonymous functions, similarly to anonymous classes used in Java.
- The facilities of CL make it possible to model and compile just about any kind of runtime semantics, including very non-linear ones, e.g. PAIP prolog.
- Prolog provides an simple way to build parsers, if efficiency is not of utmost concern. A number of prologs are available to CL programmers, so one can have the best of both worlds. [I use paip prolog to parse diagrammatic syntax.]
hth
pt
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

--
Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY

Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.

[r***@public.gmane.org]

Content preview: This may not be much of an example, but it's not only what
I have, it's what got me hooked on Lisp. When I was a young lad, and because
I lived not too far from Lake Geneva, I played this newfangled thing called
D&D (back when it first came out. Get off my lawn). And one of the things
that intrigued me was the tables (the kind where you rolled dice and looked
up the results). And also being as I was learning programming (on a mainframe.
There were no microcompputers. See my previous comment about lawns.) I thought
about writing a table language. One where I could specify not only how many
chances a particular result had (because I'd already figured out that that
was better than using the dice themselves for a bell curve), but also being
able to modify the tables themselves at run-time by feeding them arguments,
being able to have sub-tables, detailed results, etc. But at the time all
we had was Honeywell mainframe BASIC (supposedly FORTRAN was available, but
we couldn't get a manual), and even the instructor didn't know anything about
writing parsers, let alone languages. [...]

Content analysis details: (0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[64.29.147.43 listed in list.dnswl.org]
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/534>

This may not be much of an example, but it's not only what I have, it's
what got me hooked on Lisp.

When I was a young lad, and because I lived not too far from Lake
Geneva, I played this newfangled thing called D&D (back when it first
came out. Get off my lawn). And one of the things that intrigued me was
the tables (the kind where you rolled dice and looked up the results).
And also being as I was learning programming (on a mainframe. There
were no microcompputers. See my previous comment about lawns.) I
thought about writing a table language. One where I could specify not
only how many chances a particular result had (because I'd already
figured out that that was better than using the dice themselves for a
bell curve), but also being able to modify the tables themselves at
run-time by feeding them arguments, being able to have sub-tables,
detailed results, etc. But at the time all we had was Honeywell
mainframe BASIC (supposedly FORTRAN was available, but we couldn't get
a manual), and even the instructor didn't know anything about writing
parsers, let alone languages.

Fast forward, and the idea stuck with me. I mean, I'd gone to school
and doubles in ECE and Comp. Sci., I should be able to do this. But I
never did, because it's a very big hassle to write your own language.
Especially in languages like C.

So when I started using Lisp, I read up all this stuff about DSLs. Hm,
I wonder if I could use Lisp to do my language. Well, half an hour and
15 lines of code later, I not only had my language, but more than I
wanted. And here it is (ta da):

(defmacro table (name args &rest rest)
`(defun ,name (,@args)
(declare (special ,@args))
(let* ((base ',(loop for x in rest collect (car x)))
(results ',(loop for x in rest collect (cadr x)))
(chances (loop for x in base collect (eval x)))
(total (loop for x in chances sum x))
(roll (+ 1 (random total))))
;(format t "Base:~{~a~^, ~}~%Chances:~{~a~^, ~}~%Results:~{~a~^,
~}~%total:~A roll:~A~%"
; base chances results total roll)
(do ((n chances (cdr n))
(cnt 0)
(item 0 (+ item 1)))
((eq n nil) nil)
(setf cnt (+ cnt (car n)))
(when (<= roll cnt)
(return (eval (nth item results))))))))

and it's used like this (a simple example):

(table my-table (added-chances)
(1 "one")
(1 "two")
((+ 1 added-chances) "three")
)

I've done some stuff with it that's been quite a bit of fun. Like
another layer of macro on top for those who think in terms of dice.

The upshot is that it's a project I didn't do for 25 years because it
was too big a pain. And I did it in Lisp in half an hour, including
figuring out that I had to use dynamic scoping for args. I have to use
dynamic scoping because the forms being eval'ed need to be able to
access args, but args is not in their lexical scope (though it looks as
if they are in the macro invocation -- which is the point).

Neil Gilmore
raito-***@public.gmane.org

[Alessio Stalla]

Content preview: On Wed, Jul 20, 2011 at 3:32 PM, Didier Verna wrote: > > Dear
friends, > > I'm starting to write a chapter for an upcoming book on domain
specific > languages. The chapter is called (tentatively): > > Extensible
languages -- blurring the distinction between DSLs and GPLs > > GPL meaning
General Purpose Language in this context ;-) > > > My intention is to demonstrate
how the task of implementing a DSL is > made easier when it boils down to
an extension or subset of your > original GPL (hence reusing its infrastructure),
instead of being a > totally different language, only written on top of the
other. > > Obviously, I'm going to illustrate this with Common Lisp, and
I intend > to speak of dynamicity (not only dynamic typing, but in general
all > things that can be deferred to the run-time), introspection, > intersession,
structural or procedural reflexivity, meta-object > protocols (not sure about
this one), macro systems and JIT-compilation. > Also, more specifically to
Lisp, reader macros (compiler macros maybe?), > the condition system (and
its ability to *not* unwind) and restarts. > > > Right now, I would like
to know if any of you have DSL "pearls", nice > examples of DSLs that you
have written in Lisp by using some of its > features in a clever or elegant
way. I would also gladly accept any > point of view or comment on what's
important to mention, in terms of > design principle or anything else, things
that I may have missed in the > list above. [...]

Content analysis details: (-99.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
1.5 SINGLE_HEADER_2K A single header contains 2K-3K characters
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(alessiostalla[at]gmail.com)
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.216.179 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/535>

 Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.

Not really "pearls", but lately I found myself using more and more of
Lisp at macroexpansion time, which seems to be what you're looking
for. Two examples:

- for my DO+ (doplus) iteration macro - a somewhat iterate-like
construct with a simpler implementation that does not use a code
walker - I used structures quite heavily. do+ controls iteration via a
series of clauses that are macros (either built-in or written by
users) that have a peculiar aspect: they don't expand to proper Lisp
code, but return structures or lists of structures [1]. do+
macroexpands its clauses (recursively when needed) and uses the
returned structures to drive the building of its own expansion.
Basically I explicitly used macros as compile-time functions, defining
my own limited "AST" (not really a tree but a flat list of structures)
which is not Lisp's but is later translated to Lisp, so it's one extra
indirection than what is typically found in macros. I think this shows
nicely how the distinction between code and data is blurry in Lisp,
and as a consequence the distinction between DSL and GPL is blurry as
well. This allows the implementation to remain simple: do+ is a little
less than 350 lines of code, but only roughly half of them are the
actual macro and its supporting code, the rest are built-in clause
macros and other minor stuff.

- I also have sketched a HTML generation library - tentatively called
tag-pool - where I used CLOS generic functions at macroexpansion time
to drive the expansion of tag macros to HTML-outputting code. Contrary
to the popular approach, used by CL-WHO among others, to represent
HTML as a limited sexp-based DSL where tag names are keywords and a
single macro does all the translation work, I have one macro per tag.
For CLOS dispatch purposes, to each tag is also associated a class,
but it's used only at expansion time, the output code does not
instantiate any CLOS object. Tag macros follow a CLOS-based protocol
to emit the various bits of HTML like attributes, body, and so on.
Users can specialize methods that are part of the protocol to finely
control how tag macros are translated, for example providing defaults
for certain attributes, or post-processing some attribute's value,
etc. As an example, the SCRIPT macro translates its Lisp body to
JavaScript using Parenscript, by specializing a single generic
function. The body code can itself contain HTML-generating macros and
those are translated to HTML-emitting JavaScript as well. Again, a key
feature of Lisp (CLOS and generic function dispatch) is used at
macroexpansion time to make the implementation of the DSL simpler and
more easily extensible.

You can find both examples here: <http://code.google.com/p/tapulli/>
although only do+ is documented and reasonably complete.

[1] ok, technically a structure *is* proper Lisp code since it is a
self-evaluating object and it can be externalized...

Best,
Alessio

[Jean-Philippe Paradis]

Content preview: This is pretty tragic for me because I made and open-sourced
a bunch of "pretty good" DSL's, unfortunately they're still not cleaned up
and completely undocumented. Also, all the usage examples of them are buried
in some big abortive undocumented proprietary project with lots of newb-code
(most of my mastery of Common Lisp was acquired by making this project),
however I'm its sole author and own all the rights so I might consider open-sourcing
it (perhaps under the GPL or something?). I had not considered this possibility
until now mostly out of shame for the low overall quality of this code (with
some peaks of high-quality I think, which might interest you). [...]

Content analysis details: (-0.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(hexstream[at]gmail.com)
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.210.179 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/536>

This is pretty tragic for me because I made and open-sourced a bunch
of "pretty good" DSL's, unfortunately they're still not cleaned up and
completely undocumented. Also, all the usage examples of them are
buried in some big abortive undocumented proprietary project with lots
of newb-code (most of my mastery of Common Lisp was acquired by making
this project), however I'm its sole author and own all the rights so I
might consider open-sourcing it (perhaps under the GPL or something?).
I had not considered this possibility until now mostly out of shame
for the low overall quality of this code (with some peaks of
high-quality I think, which might interest you).

I don't know under what kind of time constraints you're working, it's
obvious that I won't have time to cleanup and document all of these in
time, however perhaps if I do a bit of digging in these old projects
(that I'm not even sure still build) I might be able to point you to
the right places and maybe we can dig out a few examples of "pearls"
in this mountain of shit? Who knows...

Anyway, here are some pointers.

http://paste.lisp.org/display/118464

[Alessio Stalla]

Content preview: On Wed, Jul 20, 2011 at 9:21 PM, Martin Cracauer wrote: >
I find that people have a wrong impression of what it means to have a > domain
specific language. > > For me, one of the greatest advantages of Lisp is
literal data in > code. I don't have to write a complicated parser for some
external > data, I don't have to hook up a XML reader library to a bunch
of > callbacks and construct a graph or tree or whatever in those callbacks

and then I have to write that graph in XML (ugh) with no programming >

help at all unless I pipe it through m4. In Lisp code I just write the > graph,
the entire thing, no matter how complicated. And although it > is Lisp source
code it doesn't have to live in the source > distribution, I can load an
external file later. > > And it gets to use all the Lisp tools I have built
for my application > anyway, to help me construct that complicated data structure,
without > using m4. > > That does a lof of what many people mean when they
say they need a > DSL. What they really want is not to have to do a bazillion
lines of > code to hook up the ability to read complex structured real-world

data. [...]

Content analysis details: (-99.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.216.179 listed in list.dnswl.org]
-100 USER_IN_WHITELIST From: address is in the user's white-list
1.5 SINGLE_HEADER_2K A single header contains 2K-3K characters
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(alessiostalla[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/538>

I find that people have a wrong impression of what it means to have a
domain specific language.
For me, one of the greatest advantages of Lisp is literal data in
code.  I don't have to write a complicated parser for some external
data, I don't have to hook up a XML reader library to a bunch of
callbacks and construct a graph or tree or whatever in those callbacks
and then I have to write that graph in XML (ugh) with no programming
help at all unless I pipe it through m4. In Lisp code I just write the
graph, the entire thing, no matter how complicated.  And although it
is Lisp source code it doesn't have to live in the source
distribution, I can load an external file later.
And it gets to use all the Lisp tools I have built for my application
anyway, to help me construct that complicated data structure, without
using m4.
That does a lof of what many people mean when they say they need a
DSL. What they really want is not to have to do a bazillion lines of
code to hook up the ability to read complex structured real-world
data.

I agree, that's one of the greatest advantages of DSLs in Lisp (so
called "internal" DSLs) versus "external" DSLs which seem to be
getting the most mainstream attention these days, with complex tools
to design parsers, editors and whatnot. Without even touching macros,
the Lisp reader and printer systems already provide invaluable tools
for reading and writing complex data, tools that other languages lack.
Macros can then help as a little syntactic sugar, to remove some
boilerplate, or to do some bookkeeping, or to move some computation to
compile-time (another huge win other languages completely lack). XML,
JSON, YAML etc. are so hyped because they introduce a little dynamics
in otherwise mostly static languages like Java, but they come with the
burden of libraries for parsing, writing, binding to objects, and have
trouble expressing shared structure.
Besides, in Lisp-based DSLs, it's generally easy or even trivial to
escape to Lisp when needed, to use e.g. conditionals or looping, two
basic programming language constructs that are lacking (and often
missed) in "data-oriented" languages like XML.
So, the code = data paradigm is really two-way: you can manipulate
code as data, but you can also treat data as part of the code, and the
latter point is often neglected.

Alessio

[Vsevolod Dyomkin]

I like how Lisp allowed me to use almost without any change the spec of
Redis to define its commands in cl-redis library (
https://github.com/vseloved/cl-redis/blob/master/commands.lisp). This gives,
IMHO, the best possible decoupling from the implementation, which allows for
easy refactoring of the underlying mechanisms (performed twice in the course
of the library's existence) and seamless addition of new commands.

It's a good example of what Rainer Joswig called "putting parenthesis around
the specification and making it run" (
http://lispm.dyndns.org/mov/dsl-in-lisp.mov)

Best regards,
Vsevolod

Post by Didier Verna
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!
--
Resistance is futile. You will be jazzimilated.
Scientific site: http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[Steve Haflich]

You might want to examine
https://github.com/franzinc/net-xml-generatorwhich is (sort-of) a DSL
for generating and pretty-printing XML. It uses
reader macros and the pretty-printer so that XML elements can be freely
intermixed and nested inside Lisp code. (Unlike HTML, XML has an unbounded
set of tags, so defining a finite set of keywords won't work.)
Download the zip and view the xhtml blurb in a browser. It
self-referentially contains the Lisp code that generated it, which is
probably harder to do in other languages.

[Marco Antoniotti]

Content preview: Well, since we are talking about XML stuff, I have another
shameless plug myself then :) http://within-parens.blogspot.com/2011/06/printing-xhtm.html
All in all the pretty-printer is very nice, it'd be nice to have an agreed
upon jazzed up version of CLOS dispatching. [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain

Well, since we are talking about XML stuff, I have another shameless plug myself then :)

http://within-parens.blogspot.com/2011/06/printing-xhtm.html

All in all the pretty-printer is very nice, it'd be nice to have an agreed upon jazzed up version of CLOS dispatching.

Cheers

MA

You might want to examine https://github.com/franzinc/net-xml-generator which is (sort-of) a DSL for generating and pretty-printing XML. It uses reader macros and the pretty-printer so that XML elements can be freely intermixed and nested inside Lisp code. (Unlike HTML, XML has an unbounded set of tags, so defining a finite set of keywords won't work.)
Download the zip and view the xhtml blurb in a browser. It self-referentially contains the Lisp code that generated it, which is probably harder to do in other languages.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

--
Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY

Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.

[Alexander Repenning]

XML, good point. XMLisp combines S-expressions and XML into X-expression via CLOS, MOP, Lisp reader and Lisp printer:
- paper: http://www.cs.colorado.edu/~ralex/papers/PDF/X-expressions.pdf
- code (all in one file) http://code.google.com/p/xmlisp/source/browse/trunk/XMLisp/sources/XMLisp/XMLisp.lisp
- application: http://code.google.com/p/xmlisp/

best, Alex

Post by Marco Antoniotti
Well, since we are talking about XML stuff, I have another shameless plug myself then :)
http://within-parens.blogspot.com/2011/06/printing-xhtm.html
All in all the pretty-printer is very nice, it'd be nice to have an agreed upon jazzed up version of CLOS dispatching.
Cheers
MA

You might want to examine https://github.com/franzinc/net-xml-generator which is (sort-of) a DSL for generating and pretty-printing XML. It uses reader macros and the pretty-printer so that XML elements can be freely intermixed and nested inside Lisp code. (Unlike HTML, XML has an unbounded set of tags, so defining a finite set of keywords won't work.)
Download the zip and view the xhtml blurb in a browser. It self-referentially contains the Lisp code that generated it, which is probably harder to do in other languages.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

--
Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY
Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

Prof. Alexander Repenning

University of Colorado
Computer Science Department
Boulder, CO 80309-430

vCard: http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf

[Pascal J. Bourguignon]

Content preview: Didier Verna <didier-+R3Ik2xuP/***@public.gmane.org> writes: > Dear friends,
http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/0245342429c61800/f7966bf3df9f716c#f7966bf3df9f716c
[...]

Content analysis details: (-3.5 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/540>

Post by Didier Verna
Dear friends,

http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/0245342429c61800/f7966bf3df9f716c#f7966bf3df9f716c

Notably, in this type of DSL, you'll note how you can mix Lisp
expressions with expressions in the new languages, at several levels.


@[Icals method:(pressPop:(:id)sender) ;
resultType:(:id) ; objcl
body: ;
(declare (ignore sender)) ;
(loop ; lisp
:for i :from 0 :below 3 ;
:do [(aref (icals-stack self) i) setIntValue:[(aref (icals-stack self) (1+ i)) intValue]])]
^^ ^^
|| ||
|+-- lisp |+--- lisp
+--- objcl +---- objcl

It might be worthwhile comparing the sources of an early Objective-C
front-end such as POC (that includes only the parsing of Objective-C
syntax, and generation of the C code for the usual C compiler, and thus
is functionnaly equivalent to ObjCL), with the lisp sources of ObjCL.

http://users.telenet.be/stes/compiler.html
(objc-3.2.11.tar.gz)

[Marco Antoniotti]

Content preview: Another cut example I did many years ago (showing my NYU colors)
along the lines of what Pascal shows here is the "SETL syntax" in CL. I still
have the code. Here is the stupid PRIMES (defun primes (max) [n in (range
3 max 2) such-that (not (exist m in (range 3 (min (1- n) (+ 2 (sqrt n)))
2) such-that (= (mod n m) 0)))]) [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain

Another cut example I did many years ago (showing my NYU colors) along the lines of what Pascal shows here is the "SETL syntax" in CL. I still have the code. Here is the stupid PRIMES

(defun primes (max)
[n in (range 3 max 2)
such-that (not (exist m in (range 3 (min (1- n) (+ 2 (sqrt n))) 2)
such-that (= (mod n m) 0)))])

Cheers

MA

Post by Pascal J. Bourguignon

Post by Didier Verna
Dear friends,

http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/0245342429c61800/f7966bf3df9f716c#f7966bf3df9f716c
Notably, in this type of DSL, you'll note how you can mix Lisp
expressions with expressions in the new languages, at several levels.
@[Icals method:(pressPop:(:id)sender) ;
resultType:(:id) ; objcl
body: ;
(declare (ignore sender)) ;
(loop ; lisp
:for i :from 0 :below 3 ;
:do [(aref (icals-stack self) i) setIntValue:[(aref (icals-stack self) (1+ i)) intValue]])]
^^ ^^
|| ||
|+-- lisp |+--- lisp
+--- objcl +---- objcl
It might be worthwhile comparing the sources of an early Objective-C
front-end such as POC (that includes only the parsing of Objective-C
syntax, and generation of the C code for the usual C compiler, and thus
is functionnaly equivalent to ObjCL), with the lisp sources of ObjCL.
http://users.telenet.be/stes/compiler.html
(objc-3.2.11.tar.gz)
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

--
Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY

Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.

[Daniel Pezely]

Content preview: On Jul 20, 2011, at 6:32 AM, Didier Verna wrote: > Right now,
I would like to know if any of you have DSL "pearls", nice > examples of
DSLs that you have written in Lisp by using some of its > features in a clever
or elegant way. I would also gladly accept any > point of view or comment
on what's important to mention, in terms of > design principle or anything
else, things that I may have missed in the > list above. [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[64.142.19.5 listed in list.dnswl.org]
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/544>

Post by Didier Verna
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.

One perspective that I haven't seen addressed yet is:
know your audience and related concepts.

Last year, I wrote a tool for processing large quantities of dirty .csv files: more than can fit into RAM, multiple files comprising a single dataset, column sequence changing across files, junk in cell values, different number of columns across files, etc. And extremely short turn-around time.

The idea was to process this data without having to clean it up first, such that one might gain some insights (sums, uniques, bucketing, etc.) helpful in determining whether or not the data was of any value or gain clues on how to proceed further. Proper database ETL would be too time consuming.


My intended audience was a statistical analyst that favored mousing around Excel spreadsheets while letting prior SAS training go unused because of not wanting to write scripts. (That should have been the first clue to be concerned, but oh, what we do for friends!)

Ultimately, the tool was useful to me, and that was sufficient to meet deadlines.


But the DSL code became overly complex due to this extraneous design criteria of accommodating someone who doesn't want to write scripts in the first place.

Instead of a few simple macros such as WITH-MESSY-CSV appropriate to a Lisp programmer, I effectively created a register machine with "simple" commands: include/exclude filtering, sum, count, unique, look-up, group-by, etc.

Otherwise, the approach was sound enough: load relatively small .csv files as look-up tables and iterate over the entire dataset in a single pass, applying lexically scoped blocks of filtering & calculations. Convert only named columns of interest regardless of position changes across files, parse numbers on demand only for those operations that required it, skip unparseable rows as last resort, etc. Some error in results-- but some results with reduced confidence are better than none in this case.


Lessons learned: (a few more while I'm here)

1. Know your audience, and build for the correct users.

2. Build the right tool. (I'm a systems programmer; a good stats person would likely have come up with a better work-flow, likely using R so rich reports could also be generated quickly.)

3. Good language design can be challenging. I would have been better off (perhaps) stealing SQL or XQuery's FLOWR conventions than inventing my own "simple" set of commands. (Syntax is another matter... as you know.)

4. Being adept at backquotes, comma substitution and unrolling lists is not necessarily enough skill to create a good, clean DSL implementation. But keep trying. Do your best to make one for "keeps". Then throw it away, anyway. It's important to not hold anything back in the first version. Ah, experience! (I'll likely go at this one again just for the fun of it.)
e.g., unrelated project from years ago: http://play.org/learning-lisp/html.lisp

5. Collaborate: Get input from others. My co-workers who also use Common Lisp were many time-zones and an ocean away, busy with looming deadlines of their own. However, their 10 years CL experience to my 5 (and their far deeper stats familiarity) would certainly have helped here.

-Daniel

--
first name at last name dot com

[Marco Antoniotti]

Content preview: +1 May I also say that there are entire scientific, financial,
and accounting communities that should be barred from using Excel? Cheers
-- MA [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/545>

+1

May I also say that there are entire scientific, financial, and accounting communities that should be barred from using Excel?

Cheers
--
MA

Post by Daniel Pezely
...
Lessons learned: (a few more while I'm here)
1. Know your audience, and build for the correct users.
2. Build the right tool. (I'm a systems programmer; a good stats person would likely have come up with a better work-flow, likely using R so rich reports could also be generated quickly.)
3. Good language design can be challenging. I would have been better off (perhaps) stealing SQL or XQuery's FLOWR conventions than inventing my own "simple" set of commands. (Syntax is another matter... as you know.)
4. Being adept at backquotes, comma substitution and unrolling lists is not necessarily enough skill to create a good, clean DSL implementation. But keep trying. Do your best to make one for "keeps". Then throw it away, anyway. It's important to not hold anything back in the first version. Ah, experience! (I'll likely go at this one again just for the fun of it.)
e.g., unrelated project from years ago: http://play.org/learning-lisp/html.lisp
5. Collaborate: Get input from others. My co-workers who also use Common Lisp were many time-zones and an ocean away, busy with looming deadlines of their own. However, their 10 years CL experience to my 5 (and their far deeper stats familiarity) would certainly have helped here.
-Daniel

--
Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY

Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.

[Thomas M. Hermann]

(1+ *)

I am knee-deep daily in one of those communities. Time and time again, I
request that people send me the raw text files of data as opposed to poorly
importing them into an Excel spreadsheet. Or, I'll get some spreadsheet
"analysis" that contains brittle calculations that require lots of hand
editing if you wish to update data, (select ranges, etc.). If people are
going to rely on Excel to the extent that they do, I wish they'd at least
learn VBA.

I have a large body of lisp code that is purely an abstraction layer to
insulate me from a very poorly conceived DSL for a finite element analysis
package. Having the lisp abstraction is like a productivity accelerator. The
more I have abstracted, the more productive I am and the more time I have to
implement higher level abstractions.

Tom
----------------------------------------------------------------
Thomas M. Hermann
Odonata Research LLC
http://www.odonata-research.com/
http://www.linkedin.com/in/thomasmhermann


On Fri, Jul 22, 2011 at 2:54 AM, Marco Antoniotti <

Post by Marco Antoniotti
+1
May I also say that there are entire scientific, financial, and accounting
communities that should be barred from using Excel?
Cheers
--
MA

Post by Daniel Pezely
...
Lessons learned: (a few more while I'm here)
1. Know your audience, and build for the correct users.
2. Build the right tool. (I'm a systems programmer; a good stats person

would likely have come up with a better work-flow, likely using R so rich
reports could also be generated quickly.)

Post by Daniel Pezely
3. Good language design can be challenging. I would have been better

off (perhaps) stealing SQL or XQuery's FLOWR conventions than inventing my
own "simple" set of commands. (Syntax is another matter... as you know.)

Post by Daniel Pezely
4. Being adept at backquotes, comma substitution and unrolling lists is

not necessarily enough skill to create a good, clean DSL implementation.
But keep trying. Do your best to make one for "keeps". Then throw it
away, anyway. It's important to not hold anything back in the first
version. Ah, experience! (I'll likely go at this one again just for the
fun of it.)
http://play.org/learning-lisp/html.lisp

Post by Daniel Pezely
5. Collaborate: Get input from others. My co-workers who also use

Common Lisp were many time-zones and an ocean away, busy with looming
deadlines of their own. However, their 10 years CL experience to my 5 (and
their far deeper stats familiarity) would certainly have helped here.

Post by Daniel Pezely
-Daniel

--
Marco Antoniotti, Associate Professor tel. +39
- 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043
http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY
Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[Daniel Weinreb]

If you have people making messy spreadsheets that are
hard to maintain, you might want to look at this. I have
not used it myself:

http://www.modelsheetsoft.com

But it's from Howard Cannon, Symbolics co-founder and
main inventor of Flavors, an ancestor of CLOS.

-- Dan

On Fri, Jul 22, 2011 at 9:56 AM, Thomas M. Hermann <

Post by Thomas M. Hermann
(1+ *)
I am knee-deep daily in one of those communities. Time and time again, I
request that people send me the raw text files of data as opposed to poorly
importing them into an Excel spreadsheet. Or, I'll get some spreadsheet
"analysis" that contains brittle calculations that require lots of hand
editing if you wish to update data, (select ranges, etc.). If people are
going to rely on Excel to the extent that they do, I wish they'd at least
learn VBA.
I have a large body of lisp code that is purely an abstraction layer to
insulate me from a very poorly conceived DSL for a finite element analysis
package. Having the lisp abstraction is like a productivity accelerator. The
more I have abstracted, the more productive I am and the more time I have to
implement higher level abstractions.
Tom
----------------------------------------------------------------
Thomas M. Hermann
Odonata Research LLC
http://www.odonata-research.com/
http://www.linkedin.com/in/thomasmhermann
On Fri, Jul 22, 2011 at 2:54 AM, Marco Antoniotti <

Post by Marco Antoniotti
+1
May I also say that there are entire scientific, financial, and accounting
communities that should be barred from using Excel?
Cheers
--
MA

Post by Daniel Pezely
...
Lessons learned: (a few more while I'm here)
1. Know your audience, and build for the correct users.
2. Build the right tool. (I'm a systems programmer; a good stats

person would likely have come up with a better work-flow, likely using R so
rich reports could also be generated quickly.)

Post by Daniel Pezely
3. Good language design can be challenging. I would have been better

off (perhaps) stealing SQL or XQuery's FLOWR conventions than inventing my
own "simple" set of commands. (Syntax is another matter... as you know.)

Post by Daniel Pezely
4. Being adept at backquotes, comma substitution and unrolling lists is

not necessarily enough skill to create a good, clean DSL implementation.
But keep trying. Do your best to make one for "keeps". Then throw it
away, anyway. It's important to not hold anything back in the first
version. Ah, experience! (I'll likely go at this one again just for the
fun of it.)
http://play.org/learning-lisp/html.lisp

Post by Daniel Pezely
5. Collaborate: Get input from others. My co-workers who also use

Common Lisp were many time-zones and an ocean away, busy with looming
deadlines of their own. However, their 10 years CL experience to my 5 (and
their far deeper stats familiarity) would certainly have helped here.

Post by Daniel Pezely
-Daniel

--
Marco Antoniotti, Associate Professor tel. +39
- 02 64 48 79 01
DISCo, Università Milano Bicocca U14 2043
http://bimib.disco.unimib.it
Viale Sarca 336
I-20126 Milan (MI) ITALY
Please note that I am not checking my Spam-box anymore.
Please do not forward this email without asking me first.
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[Vsevolod Dyomkin]

Speaking of XML, there's another good example of a special-purpose DSL,
designed for efficient processing of OSM map data, described here:
http://swizard.livejournal.com/142027.html (text in Russian).

Post by Didier Verna
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is
made easier when it boils down to an extension or subset of your
original GPL (hence reusing its infrastructure), instead of being a
totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend
to speak of dynamicity (not only dynamic typing, but in general all
things that can be deferred to the run-time), introspection,
intersession, structural or procedural reflexivity, meta-object
protocols (not sure about this one), macro systems and JIT-compilation.
Also, more specifically to Lisp, reader macros (compiler macros maybe?),
the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice
examples of DSLs that you have written in Lisp by using some of its
features in a clever or elegant way. I would also gladly accept any
point of view or comment on what's important to mention, in terms of
design principle or anything else, things that I may have missed in the
list above.
Thank you very much in advance!
--
Resistance is futile. You will be jazzimilated.
Scientific site: http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com
_______________________________________________
pro mailing list
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro

[r***@public.gmane.org]

Content preview: Or at least learn the difference between a spreadsheet and
a database... I see too many spreadsheets used as databases. Neil Gilmore
raito-***@public.gmane.org [...]

Content analysis details: (0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[64.29.147.32 listed in list.dnswl.org]
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/549>

Or at least learn the difference between a spreadsheet and a database...

I see too many spreadsheets used as databases.

Neil Gilmore
raito-***@public.gmane.org

Quoting "Thomas M. Hermann" <thomas.m.hermann-vwvrpGovRlrta4EC/59zMFaTQe2KTcn/@public.gmane.org>:

If people are going to rely on Excel to the extent that they do, I wish
they'd at least learn VBA.

[Ben Hyde]

Content preview: Is there an excel-hater's mailing list we could move this
discussion to. :) ps - Is it possible to open the slime inspector on a stack
frame or frames? pro mailing list pro-***@public.gmane.org http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro
[...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[208.72.237.35 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/550>

Is there an excel-hater's mailing list we could move this discussion
to. :)

ps - Is it possible to open the slime inspector on a stack frame or
frames?

[Didier Verna]

Content preview: Thank you all very much for the feedback. I guess I have
a lot to read for the holidays now... -- Resistance is futile. You will be
jazzimilated. Scientific site: http://www.lrde.epita.fr/~didier Music (Jazz)
site: http://www.didierverna.com [...]

Content analysis details: (-100.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/551>


Thank you all very much for the feedback. I guess I have a lot to read
for the holidays now...

[Ala'a Mohammad]

Content preview: [oops, I keep forgetting to add cl-pro email when replying.]
IMHO: DSL can be made of a varying mix of abstraction, syntax and semantics.
All of which can be achieved within CL using interpreter & compiler, macros,
closures, reader macros, compiler macros, printing protocol and CLOS. These
will help a lot, both the developer and the user. [...]

Content analysis details: (-0.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(amalawi[at]gmail.com)
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.212.54 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/552>

[oops, I keep forgetting to add cl-pro email when replying.]

IMHO:

DSL can be made of a varying mix of abstraction, syntax and semantics.
All of which can be achieved within CL using interpreter & compiler,
macros, closures, reader macros, compiler macros, printing protocol
and CLOS. These will help a lot,  both the developer and the user.

check the following examples:

Corman CL Assembler DSL (check the source code for assembler.lisp and
how elegantly normal x86 assembler syntax was introduced into CL code)
http://www.cormanlisp.com/index.html

SBCL VOPs:
http://sbcl-internals.cliki.net/Adding%20VOPs
http://paste.lisp.org/display/41937

GOAL: Game Object Assembly Lisp ("...intermix assembly instructions
seamlessly with higher-level code...")
http://www.gamasutra.com/view/feature/2985/postmortem_naughty_dogs_jak_and_.php?print=1

CLSQL: SQL syntax

CL-Typesetting: typesetting DSL

Prolog interpreter or compiler.

those are the ones I remember directly when talking about DSL in common-lisp

- Ala'a

[Matthew Mondor]

Content preview: On Thu, 28 Jul 2011 15:19:13 +0400 "Ala'a Mohammad" <amalawi-***@public.gmane.org>
wrote: > those are the ones I remember directly when talking about DSL in
common-lisp Perhaps also of interest is CL-PPCRE, as if I remember it also
allows access to the internal parse tree, and provides easy means to create/manipulate
them... -- Matt [...]

Content analysis details: (-1.2 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-1.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/553>

On Thu, 28 Jul 2011 15:19:13 +0400

Post by Ala'a Mohammad
those are the ones I remember directly when talking about DSL in common-lisp

Perhaps also of interest is CL-PPCRE, as if I remember it also allows
access to the internal parse tree, and provides easy means to
create/manipulate them...