User defined format functions

[Didier Verna]

Hello,

there are situations in which I find myself frequently using the same
format "pattern", which I'd like to abstract away. A concrete example is
from my Declt package[1] which prints a lot in Texinfo format.

Since Texinfo has a couple of special characters (e.g. @), I have an
ESCAPE function which takes a string and returns another one with all
special characters escaped. In the Declt code, I hence have very
frequent calls which look like this:

(format stream "... ~A ..." #|...|# (escape str) #|...|#)


This is frustrating because I would like to abstract away the relation
"~A <-> #'escape". Of course there are solutions (I could provide my own
wrapper around FORMAT in some way) but none of them seem satisfactory.

I'm aware of the ~// construct, which I find extremely cumbersome. Does
anybody actually use it? It seems to me that the package handling part,
specifically, makes it totally unusable. Just imagine that I would need
to do something like that if I were to use it:

(format stream "... ~/com.dvlsoft.declt:escape/ ..." #|...|# str #|...|#)

and you get the idea! :-)


No, in fact, the problem is that FORMAT control sequences are not
extensible. This reminds me of something that we have in the Gnus
mail/newsreader. It's called "user format functions". Gnus Group and
Summary buffer lines, for example, are define by a format string with a
set of predefined %<char> constructs. One of them, %u<char> lets you
define and use your own function, called
gnus-user-format-function-<char>.


So what I would like to do is more or less define a function
FORMAT-FUNCTION-E, just like a standard formatter, and then be able to
write something like that:

(format stream "... ~\e\ ..." #|...|# str #|...|#)


Of course, there are package issues. Maybe we would need a centralized
mapping between function names (what goes in ~\\) and actual symbols
denoting the actual function? Possibly with the possibility to override
the mapping by providing a package prefix in the ~\\ construct?

In fact, the more I think about it, the more it sounds like having a
format-table facility, much like what a readtable is, with a similar
API. You could do (in-format-table :my-table-name) at the beginning of a
file and have a totally customized set of format control sequences,
possibly inheriting the standard ones.



WDYT? Does something like that already exists[2]?



Footnotes:
[1] http://www.lrde.epita.fr/~didier/software/lisp/misc.php#declt

[2] I've heard of something called fmt for scheme, but don't know much
about it.

[Didier Verna]

Hello,

there are situations in which I find myself frequently using the same
format "pattern", which I'd like to abstract away. A concrete example is
from my Declt package[1] which prints a lot in Texinfo format.

Since Texinfo has a couple of special characters (e.g. @), I have an
ESCAPE function which takes a string and returns another one with all
special characters escaped. In the Declt code, I hence have very
frequent calls which look like this:

(format stream "... ~A ..." #|...|# (escape str) #|...|#)


This is frustrating because I would like to abstract away the relation
"~A <-> #'escape". Of course there are solutions (I could provide my own
wrapper around FORMAT in some way) but none of them seem satisfactory.

I'm aware of the ~// construct, which I find extremely cumbersome. Does
anybody actually use it? It seems to me that the package handling part,
specifically, makes it totally unusable. Just imagine that I would need
to do something like that if I were to use it:

(format stream "... ~/com.dvlsoft.declt:escape/ ..." #|...|# str #|...|#)

and you get the idea! :-)


No, in fact, the problem is that FORMAT control sequences are not
extensible. This reminds me of something that we have in the Gnus
mail/newsreader. It's called "user format functions". Gnus Group and
Summary buffer lines, for example, are define by a format string with a
set of predefined %<char> constructs. One of them, %u<char> lets you
define and use your own function, called
gnus-user-format-function-<char>.


So what I would like to do is more or less define a function
FORMAT-FUNCTION-E, just like a standard formatter, and then be able to
write something like that:

(format stream "... ~\e\ ..." #|...|# str #|...|#)


Of course, there are package issues. Maybe we would need a centralized
mapping between function names (what goes in ~\\) and actual symbols
denoting the actual function? Possibly with the possibility to override
the mapping by providing a package prefix in the ~\\ construct?

In fact, the more I think about it, the more it sounds like having a
format-table facility, much like what a readtable is, with a similar
API. You could do (in-format-table :my-table-name) at the beginning of a
file and have a totally customized set of format control sequences,
possibly inheriting the standard ones.



WDYT? Does something like that already exists[2]?



Footnotes:
[1] http://www.lrde.epita.fr/~didier/software/lisp/misc.php#declt

[2] I've heard of something called fmt for scheme, but don't know much
about it.

[Daniel Weinreb]

Post by Didier Verna
I'm aware of the ~// construct, which I find extremely cumbersome. Does
anybody actually use it?

Yes, we use it heavily, mainly for date/time processing,
which can be very complicated when you're dealing
with time zones and such. Example:

(list :departure-time (format nil "~/zul%ISO8601/"
departure-time-zul)
:arrival-time (format nil "~/zul%ISO8601/"
arrival-time-zul)
:airimp-departure-time
(format nil "~/loc%HHMM/"
(local-tofd-only (local-time
departure-time-zul)))
:airimp-arrival-time

Post by Didier Verna
It seems to me that the package handling part,
specifically, makes it totally unusable.

You mean the fact that the name must be in the cl-user
package, so that if there are two modules trying to
use the same name, they conflict. That's true, but
I think "totally unusable" is going a bit far.

I actually can't remember why package prefixes
aren't allowed. It was a long time ago.

-- Dan

[Edi Weitz]

Post by Daniel Weinreb
I actually can't remember why package prefixes
aren't allowed.  It was a long time ago.

They are allowed, aren't they?

http://www.lispworks.com/documentation/HyperSpec/Body/22_ced.htm

[Peter Seibel]

Of course they are. That's what allows this hack (first suggested by
Erik Naggum, I believe):

(defpackage :iso (:use) (:export :|8601|))

(defun iso:8601 (out arg colon-p at-sign-p &rest params)
(write-string (format-iso-8601-time
arg
:time-zone (first params)
:omit-time (and at-sign-p (not colon-p))
:omit-date colon-p
:omit-time-zone (and colon-p (not at-sign-p))) out))


And then:

CL-USER> (format t "~/iso:8601/" (get-universal-time))
2011-01-25T11:51:15-8:00
NIL

-Peter

Post by Edi Weitz

Post by Daniel Weinreb
I actually can't remember why package prefixes
aren't allowed.  It was a long time ago.

They are allowed, aren't they?
http://www.lispworks.com/documentation/HyperSpec/Body/22_ced.htm
_______________________________________________
pro mailing list
http://common-lisp.net/cgi-bin/mailman/listinfo/pro

[Didier Verna]

Post by Peter Seibel
CL-USER> (format t "~/iso:8601/" (get-universal-time))
2011-01-25T11:51:15-8:00
NIL

Nice :-)

[Didier Verna]

Post by Edi Weitz

I actually can't remember why package prefixes aren't allowed.  It
was a long time ago.

They are allowed, aren't they?
http://www.lispworks.com/documentation/HyperSpec/Body/22_ced.htm

They are. The cl-user package is only used when the function name in
the format string doesn't have a package prefix.

[Daniel Weinreb]

Oh, sorry; right, it's just for convenience to put
them in cl-user, to keep the format strings short.

Post by Edi Weitz

Post by Daniel Weinreb
I actually can't remember why package prefixes
aren't allowed. It was a long time ago.

They are allowed, aren't they?
http://www.lispworks.com/documentation/HyperSpec/Body/22_ced.htm

[Didier Verna]

Yes, we use it heavily, mainly for date/time processing, which can be
very complicated when you're dealing with time zones and such.
(list :departure-time (format nil "~/zul%ISO8601/" departure-time-zul)

OK, but does it really buy you anything, compared to just calling a
specific function directly? Your examples all seem to contain only one
format directive.

:arrival-time (format nil "~/zul%ISO8601/" arrival-time-zul)

BTW, that should be (+ arrival-time-zul 1) for Air France :-)

It seems to me that the package handling part, specifically, makes it
totally unusable.

You mean the fact that the name must be in the cl-user package, so
that if there are two modules trying to use the same name, they
conflict. That's true, but I think "totally unusable" is going a bit
far.

I reckon it is in general. It's just that if you're writing a library
(as opposed to a top-level application), then you don't want to pollute
the cl-user package (in fact, you just can't if you want to be on the
safe side), so even when your code is (in-package :long.package.name),
you still need to use the package prefix in the format string, and
/that/, I find totally unusable.

[Daniel Weinreb]

Post by Didier Verna

Yes, we use it heavily, mainly for date/time processing, which can be
very complicated when you're dealing with time zones and such.
(list :departure-time (format nil "~/zul%ISO8601/" departure-time-zul)

OK, but does it really buy you anything, compared to just calling a
specific function directly? Your examples all seem to contain only one
format directive.

Yes, I just realized that these aren't the best examples.
They are intended to be used inside more complex
format strings, and then the developers get used
to using the format directives rather than learning
the names of the corresponding functions, just
to have one less thing to learn.

This is a specifier for a scheduled flight:

(format nil "F,~A,~A,~@[~A~],~/loc%YYYY-MM-DD/"
(sched:flight-key-carrier key)
(sched:flight-key-number key)
(sched:non-blank-operational-suffix
(sched:flight-key-op-suffix key))
(sched:flight-key-date-local key)))

Post by Didier Verna
I reckon it is in general. It's just that if you're writing a library
(as opposed to a top-level application), then you don't want to pollute
the cl-user package (in fact, you just can't if you want to be on the
safe side), so even when your code is (in-package :long.package.name),
you still need to use the package prefix in the format string, and
/that/, I find totally unusable.

Well, we find it very useful. I think there is no need
to prolong this part of the conversation; I think
everyone here is a pro and can understand the
plusses and minuses.

-- Dan

[Matthew D. Swank]

Post by Didier Verna

Yes, we use it heavily, mainly for date/time processing, which can be
very complicated when you're dealing with time zones and such.
(list :departure-time (format nil "~/zul%ISO8601/" departure-time-zul)

OK, but does it really buy you anything, compared to just calling a
specific function directly? Your examples all seem to contain only one
format directive.

Even a single directive is helpful if it's short enough. I try to
mitigate some of the more cumbersome aspects of using ~// by defining a
single generic function, format-value, that has a ~// compatible
argument list. There is an in-lined wrapper function, f, that resides
in a package that has a single letter nickname (of course that invites
other problems). Anyway, I end up using ~/p:f/ in a lot of format
control strings.

Matt

[Pascal J. Bourguignon]

Post by Didier Verna
(format stream "... ~\e\ ..." #|...|# str #|...|#)

\ is already meaningful in strings, to escape \ or ".
So you'd have to use ~\\e\\.

Post by Didier Verna
Of course, there are package issues. Maybe we would need a centralized
mapping between function names (what goes in ~\\) and actual symbols
denoting the actual function? Possibly with the possibility to override
the mapping by providing a package prefix in the ~\\ construct?

I don't like it.

Post by Didier Verna
WDYT? Does something like that already exists[2]?

The standard already allows control-strings to be functions.
So you could write a cl:formatter-like macro with any extension you
like.

There's no need for any extension, just use CL.

[Didier Verna]

\ is already meaningful in strings, to escape \ or ". So you'd have to
use ~\\e\\.

Yeah bad example. It just popped up in my mind as the opposite of ~\\.
Make that ~!! or whatever.

The standard already allows control-strings to be functions. So you
could write a cl:formatter-like macro with any extension you like.
There's no need for any extension, just use CL.

I never said I needed a real extension. I can provide a library with
an alternate (extended ;-) format function. My point was rather to share
the idea of format-tables, which you don't seem to like, whatever the
way they would be implemented.

[Nick Levine]

Yes, it is indeed quite revolting, and lacking in aesthetics. But I
find that I am using it, for some of my dynamic menu building. Oh vile
hack!

- nick

[Sam Steingold]

Post by Didier Verna
I'm aware of the ~// construct, which I find extremely cumbersome.
It seems to me that the package handling part,
specifically, makes it totally unusable.

exactly.

Post by Didier Verna
Does anybody actually use it?

yes.

Post by Didier Verna
WDYT? Does something like that already exists[2]?

clisp has "~!"

http://clisp.sourceforge.net/impnotes/print-formatted.html

The additional FORMAT instruction ~! is similar to ~/, but avoids
putting a function name into a string, thus, you might not need to
specify the package explicitly.

(FORMAT stream "~arguments!" function object)

is equivalent to

(FUNCALL function stream object colon-modifier-p atsign-modifier-p arguments)

[Didier Verna]

Post by Sam Steingold
clisp has "~!"
http://clisp.sourceforge.net/impnotes/print-formatted.html
The additional FORMAT instruction ~! is similar to ~/, but avoids
putting a function name into a string, thus, you might not need to
specify the package explicitly.
(FORMAT stream "~arguments!" function object)
is equivalent to
(FUNCALL function stream object colon-modifier-p atsign-modifier-p arguments)

That's nice. Thanks for the pointer.