why :key arguments?

Content preview: On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp wrote: > Why do
some CL library functions have :key arguments? [...] > but it is a bit cumbersome.
I can make my code simpler by relying on > calls like > > (quantiles (map
'vector key vector) quantiles) [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.210.179 listed in list.dnswl.org]
-100 USER_IN_WHITELIST From: address is in the user's white-list
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(hans.huebner[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/486>

Post by Tamas Papp
Why do some CL library functions have :key arguments?

[...]

Post by Tamas Papp
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

This not only conses "a bit more", it also duplicates traversal
efforts - The original list must be traversed, and the consed-up list
of key values as well. I think it is prudent that the CL library
functions offer ways to reduce consing for cases where "a bit" is too
much (and "a bit" can become a lot if a program operates on long
lists).

-Hans

Tamas Papp

2011-07-04 09:57:10 UTC

Post by Hans HÃ¼bner
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp

Post by Tamas Papp
Why do some CL library functions have :key arguments?

[...]

Post by Tamas Papp
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

This not only conses "a bit more", it also duplicates traversal efforts
- The original list must be traversed, and the consed-up list of key
values as well. I think it is prudent that the CL library functions
offer ways to reduce consing for cases where "a bit" is too much (and "a
bit" can become a lot if a program operates on long lists).

I understand this. My main question is: why not do this with compiler
macros? Is there any reason for this, other than historical?

Note that I am not complaining about the standard, I just want to learn
the reason for this design choice so that I can take it into account when
writing my own libraries.

Best,

Tamas

Stas Boukarev

2011-07-04 10:20:32 UTC

Post by Hans HÃ¼bner
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp

Post by Tamas Papp
Why do some CL library functions have :key arguments?

[...]

Post by Tamas Papp
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

This not only conses "a bit more", it also duplicates traversal efforts
- The original list must be traversed, and the consed-up list of key
values as well. I think it is prudent that the CL library functions
offer ways to reduce consing for cases where "a bit" is too much (and "a
bit" can become a lot if a program operates on long lists).

I understand this. My main question is: why not do this with compiler
macros? Is there any reason for this, other than historical?

Because it's not easy to do with compiler macros.

Post by Tamas Papp
Note that I am not complaining about the standard, I just want to learn
the reason for this design choice so that I can take it into account when
writing my own libraries.

I find (foo sequence :key #'key) much nicer than
(foo (map 'sequence #'key sequence))

--
With best regards, Stas.

Tamas Papp

2011-07-04 11:01:39 UTC

Post by Stas Boukarev

Post by Hans HÃ¼bner

Post by Tamas Papp
Why do some CL library functions have :key arguments?

[...]

Post by Tamas Papp
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

I understand this. My main question is: why not do this with compiler
macros? Is there any reason for this, other than historical?

Because it's not easy to do with compiler macros.

Can you (or someone) please elaborate on that? I have just started
reading up on compiler macros, and I don't understand why.

Thanks,

Tamas

Stas Boukarev

2011-07-04 11:46:07 UTC

Post by Stas Boukarev

Post by Hans HÃ¼bner

Post by Tamas Papp
Why do some CL library functions have :key arguments?

[...]

Post by Tamas Papp
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

I understand this. My main question is: why not do this with compiler
macros? Is there any reason for this, other than historical?

Because it's not easy to do with compiler macros.

Can you (or someone) please elaborate on that? I have just started
reading up on compiler macros, and I don't understand why.

Suppose you have a function SUM,

(defun sum (list)
(loop for i in list
sum i))

Now if you want to optimize (sum (map 'list KEY list)), you would
need to write an additional function.

(defun sum-key (list key)
(loop for i in list
sum (funcall key i)))

And then you would need to write a compiler macro, which would match
(map 'list KEY sequence) and transform it into (sum-key sequence KEY),
which is not that hard, if that's only what you have. Now what if you
want it to work with (mapcar KEY list) too. It becomes more and more
cumbersome as it gets more general.

And now you need two functions, a clever compiler macro (perhaps using
some pattern matching library), and which may not do what the user
expects.

While you could have all that with just one function:

(defun sum (list &key key)
(loop for i in list
sum (if key
(funcall key i)
i)))
Other disadvantages of compiler-macros:

* They are not guaranteed to be expanded. Some implementations may
ignore them.
* They can't be used with APPLY or FUNCALL.
* You can't compose them
e.g. if you wanted to write
(defun two-sum (list1 list2)
(+ (sum list1) (sum list2)))
You would need to write a second compiler macro. While with KEY
keyword you could just pass it along.

--
With best regards, Stas.

Nikodemus Siivola

2011-07-04 12:52:58 UTC

* They can't be used with APPLY or FUNCALL. Actually, they can be used

with FUNCALL. [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.210.51 listed in list.dnswl.org]
-100 USER_IN_WHITELIST From: address is in the user's white-list
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/495>

* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.

(Otherwise, I echo pretty much everything that Stas said.)

Compiler-macros are, however, of decent tool for optimizing common
cases of :KEY and :TEST arguments.

Cheers,

-- nikodemus

Pascal J. Bourguignon

2011-07-04 13:09:29 UTC

Content preview: Nikodemus Siivola <nikodemus-/W93PBX4W+***@public.gmane.org> writes: > On
4 July 2011 14:46, Stas Boukarev <stassats-***@public.gmane.org> wrote: > >> * They
can't be used with APPLY or FUNCALL. > > Actually, they can be used with FUNCALL.

(Otherwise, I echo pretty much everything that Stas said.) > > Compiler-macros

are, however, of decent tool for optimizing common > cases of :KEY and :TEST
arguments. [...]

Content analysis details: (-2.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/496>

* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
Compiler-macros are, however, of decent tool for optimizing common
cases of :KEY and :TEST arguments.

Remember however, that you cannot write in a conforming program a
compiler macro for a function named in the CL package.

clhs define-compiler-macro: "The consequences of writing a compiler
macro definition for a function in the COMMON-LISP package are
undefined;"

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Alessio Stalla

2011-07-04 13:11:30 UTC

Post by Stas Boukarev

* They can't be used with APPLY or FUNCALL. >> >> Actually, they can

be used with FUNCALL. >> >> (Otherwise, I echo pretty much everything that
Stas said.) >> >> Compiler-macros are, however, of decent tool for optimizing
common >> cases of :KEY and :TEST arguments. > > Remember however, that you
cannot write in a conforming program a > compiler macro for a function named
in the CL package. > > clhs define-compiler-macro: "The consequences of writing
a compiler > macro definition for a function in the COMMON-LISP package are

undefined;" [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(alessiostalla[at]gmail.com)
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.216.179 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/497>

On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon

Post by Stas Boukarev
* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
Compiler-macros are, however, of decent tool for optimizing common
cases of :KEY and :TEST arguments.

Remember however, that you cannot write in a conforming program a
compiler macro for a function named in the CL package.
clhs define-compiler-macro: "The consequences of writing a compiler
macro definition for a function in the COMMON-LISP package are
undefined;"

You, the user, can't, but Nikodemus, the implementer, can! ;)

Nikodemus Siivola

2011-07-04 15:58:52 UTC

Post by Pascal J. Bourguignon

Content preview: On 4 July 2011 16:11, Alessio Stalla <alessiostalla-***@public.gmane.org>
wrote: > On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon > <pjb-jNDFPZUTrfRkIYSJMME8NAC/***@public.gmane.org>
wrote: >> Nikodemus Siivola <nikodemus-/W93PBX4W+***@public.gmane.org> >> writes: >> >>>
On 4 July 2011 14:46, Stas Boukarev <stassats-***@public.gmane.org> wrote: [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.210.51 listed in list.dnswl.org]
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/501>

Post by Alessio Stalla
On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon

Post by Stas Boukarev
* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)

clhs define-compiler-macro: "The consequences of writing a compiler
macro definition for a function in the COMMON-LISP package are
undefined;"

You, the user, can't, but Nikodemus, the implementer, can! ;)

There's a misunderstanding here. I didn't mean -- nor do I think Stas
meant -- writing a compiler-macro for APPLY or FUNCALL, but rather
having a compiler-macro take effect when the function in question is
called using FUNCALL or APPLY:

(apply #'has-a-compiler-macro ...) ; compiler-macro will not fire

(funcall #'has-a-compiler-macro ...) ; compiler-macro should (if
supported) fire

Cheers,

-- nikodemus

Pascal J. Bourguignon

2011-07-04 16:42:14 UTC

Post by Pascal J. Bourguignon

* They can't be used with APPLY or FUNCALL. >>>> >>>> Actually, they can
be used with FUNCALL. >>>> >>>> (Otherwise, I echo pretty much everything
that Stas said.) > >>> clhs define-compiler-macro: "The consequences of writing
a compiler >>> macro definition for a function in the COMMON-LISP package
are >>> undefined;" >> >> You, the user, can't, but Nikodemus, the implementer,
can! ;) > > There's a misunderstanding here. I didn't mean -- nor do I think
Stas > meant -- writing a compiler-macro for APPLY or FUNCALL, but rather

having a compiler-macro take effect when the function in question is >

called using FUNCALL or APPLY: > > (apply #'has-a-compiler-macro ...) ; compiler-macro
will not fire > > (funcall #'has-a-compiler-macro ...) ; compiler-macro should
(if supported) fire [...]

Content analysis details: (-2.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/502>

Post by Alessio Stalla
On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon

Post by Stas Boukarev
* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)

clhs define-compiler-macro: "The consequences of writing a compiler
macro definition for a function in the COMMON-LISP package are
undefined;"

You, the user, can't, but Nikodemus, the implementer, can! ;)

There's a misunderstanding here. I didn't mean -- nor do I think Stas
meant -- writing a compiler-macro for APPLY or FUNCALL, but rather
having a compiler-macro take effect when the function in question is
(apply #'has-a-compiler-macro ...) ; compiler-macro will not fire
(funcall #'has-a-compiler-macro ...) ; compiler-macro should (if supported) fire

I don't think so.

First you're discussing here of a (function f), not (quote f),
and COMPILER-MACRO-FUNCTION takes a function name, not a function or
function designator. So even if the compiler did the optimization of
avoiding the call to CL:FUNCALL, it could still have difficulty and
scruples of building a source form from a function object.

But even if we considered:

(funcall (quote has-a-compiler-macro) args...)

the compiler just could not call the compiler macro of that function
name, for this reason:

(defun f (args...) (do-something args...))

(define-compiler-macro f (&environment env &whole form args...)
(if (special-f-case-p env form)
(generate-special-f-form env form args...)
form))

(flet ((f (args...) (do-something-else args...)))
(funcall (quote f) args...))

The funcall doesn't refer the same function as if we had written:

(f args...)

and therefore we cannot pass that form to the compiler macro, because it
doesn't correspond to the function of the compiler macro!

[The case of (function f) is even more telling in this case:

(flet ((f (args...) (do-something-else args...)))
(funcall (function f) args...))

since in that case, the function denoted has no compiler macro!]

And even if the implementation supports compiler macros, there's no
reason it should implement the optimization of avoiding the call to
funcall.

Therefore:

1- compiler macros won't apply when the function is called by
FUNCALL as well as when it's called by APPLY.

2- in my opinion, even in presence of a highly optimizing compiler,
synthesizing source code for the benefit of macros or compiler macros
would be a bad idea and rarely applicable without problems.

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Nikodemus Siivola

2011-07-04 17:11:56 UTC

Content preview: On 4 July 2011 19:42, Pascal J. Bourguignon <pjb-jNDFPZUTrfRkIYSJMME8NAC/***@public.gmane.org>
wrote: > I don't think so. I disagree strongly, and I'm pretty sure CLHS
agrees with me, since it goes to the trouble of specifying what happens with
FUNCALL. [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.210.51 listed in list.dnswl.org]
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/503>

Post by Pascal J. Bourguignon
I don't think so.

I disagree strongly, and I'm pretty sure CLHS agrees with me, since it
goes to the trouble of specifying what happens with FUNCALL.

CLHS, DEFINE-COMPILER-MACRO: "The &whole argument is bound to the form
argument that is passed to the compiler macro function. The remaining
lambda-list parameters are specified as if this form contained the
function name in the car and the actual arguments in the cdr, but if
the car of the actual form is the symbol funcall, then the
destructuring of the arguments is actually performed using its cddr
instead."

(I'm not really interested in fencing re. compiler-macros here, just
trying to keep the record only moderately crooked. This is a sidetrack
of epic proportions already: the OP asked about :KEY, not
compiler-macros.)

Cheers,

-- nikodemus

Tamas Papp

2011-07-05 09:51:16 UTC

On 4 July 2011 19:42, Pascal J. Bourguignon > wrote: > >> I don't think

so. > > I disagree strongly, and I'm pretty sure CLHS agrees with me, since
it > goes to the trouble of specifying what happens with FUNCALL. > > CLHS,
DEFINE-COMPILER-MACRO: "The &whole argument is bound to the form > argument
that is passed to the compiler macro function. The remaining > lambda-list
parameters are specified as if this form contained the > function name in
the car and the actual arguments in the cdr, but if the > car of the actual
form is the symbol funcall, then the destructuring of > the arguments is
actually performed using its cddr instead." > > (I'm not really interested
in fencing re. compiler-macros here, just > trying to keep the record only
moderately crooked. This is a sidetrack > of epic proportions already: the
OP asked about :KEY, not > compiler-macros.) [...]

Content analysis details: (-2.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(tkpapp[at]gmail.com)
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_TO_NO_BRKTS_FREEMAIL To: misformatted and free email service
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/504>

On 4 July 2011 19:42, Pascal J. Bourguignon

Post by Pascal J. Bourguignon
I don't think so.

I disagree strongly, and I'm pretty sure CLHS agrees with me, since it
goes to the trouble of specifying what happens with FUNCALL.
CLHS, DEFINE-COMPILER-MACRO: "The &whole argument is bound to the form
argument that is passed to the compiler macro function. The remaining
lambda-list parameters are specified as if this form contained the
function name in the car and the actual arguments in the cdr, but if the
car of the actual form is the symbol funcall, then the destructuring of
the arguments is actually performed using its cddr instead."
(I'm not really interested in fencing re. compiler-macros here, just
trying to keep the record only moderately crooked. This is a sidetrack
of epic proportions already: the OP asked about :KEY, not
compiler-macros.)

I am very happy to learn about these things. Currently I am working
on the algorithms and my main concern is to ensure correctness; speed
is secondary at this point, but even though I am not optimizing, I
want to keep my code optimizable later on.

My problem with the key argument is that it complicates the interface. I
would like to use the same interface for sample statistics and random
variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have

(mean #(1d0 2d0 3d0)) ; => 2, a sample mean
(mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution

If I had a :KEY argument, I would have to check that it is EQ to
#'identity or not provided in methods for random variables.

APPLY is not a major concern for me at the moment, all of these
functions have a fixed number of arguments (usually one or two). So
compiler macros still look attractive: I guess I could just write them
for the function I define (eg MAP1), with the understanding that if
the user wants speed, he should stick to mapping with this function.

I also thought of the following possibility using runtime dispatch:

(defstruct (w/key (:constructor w/key (key object)))
key object)

(defgeneric mean (object)
(:method ((obj w/key))
(mean-w/key (w/key-object obj) (w/key-key obj)))
(:method ((obj sequence))
(/ (reduce #'+ obj) (length obj))))

(defmethod mean-w/key ((obj sequence) key)
(/ (reduce #'+ obj :key key) (length obj)))

(mean #(1 2 3)) ; => 2
(mean (w/key #'1+ #(1 2 3))) ; => 3

Tamas

Marco Antoniotti

2011-07-05 10:03:34 UTC

Content preview: On Jul 5, 2011, at 11:51 , Tamas Papp wrote: > I am very happy
to learn about these things. Currently I am working > on the algorithms and
my main concern is to ensure correctness; speed > is secondary at this point,
but even though I am not optimizing, I > want to keep my code optimizable
later on. > > My problem with the key argument is that it complicates the
interface. I > would like to use the same interface for sample statistics
and random > variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have

(mean #(1d0 2d0 3d0)) ; => 2, a sample mean > (mean (r-normal 2 1)) ;

=> 2d0, mean of a univariate normal distribution > > If I had a :KEY argument,
I would have to check that it is EQ to > #'identity or not provided in methods
for random variables. [...]

Content analysis details: (-0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/505>

Post by Tamas Papp
I am very happy to learn about these things. Currently I am working
on the algorithms and my main concern is to ensure correctness; speed
is secondary at this point, but even though I am not optimizing, I
want to keep my code optimizable later on.
My problem with the key argument is that it complicates the interface. I
would like to use the same interface for sample statistics and random
variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have
(mean #(1d0 2d0 3d0)) ; => 2, a sample mean
(mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution
If I had a :KEY argument, I would have to check that it is EQ to
#'identity or not provided in methods for random variables.

But this is exactly where compiler macros can help. With the &key argument you keep a consistent (and useful) interface. The check whether to do away with a possible IDENTITY can be done in an appropriate compiler-macro.

Post by Tamas Papp
APPLY is not a major concern for me at the moment, all of these
functions have a fixed number of arguments (usually one or two). So
compiler macros still look attractive: I guess I could just write them
for the function I define (eg MAP1), with the understanding that if
the user wants speed, he should stick to mapping with this function.
(defstruct (w/key (:constructor w/key (key object)))
key object)
(defgeneric mean (object)
(:method ((obj w/key))
(mean-w/key (w/key-object obj) (w/key-key obj)))
(:method ((obj sequence))
(/ (reduce #'+ obj) (length obj))))
(defmethod mean-w/key ((obj sequence) key)
(/ (reduce #'+ obj :key key) (length obj)))
(mean #(1 2 3)) ; => 2
(mean (w/key #'1+ #(1 2 3))) ; => 3

You can have your cake and eat it too. Why limit yourself?

Cheers
--
Marco

Pascal J. Bourguignon

2011-07-05 11:41:42 UTC

Content preview: Tamas Papp <tkpapp-***@public.gmane.org> writes: > My problem with the
key argument is that it complicates the interface. I > would like to use
the same interface for sample statistics and random > variables, eg currently
in CL-NUM-UTILS and CL-RANDOM I have [...]

Content analysis details: (-2.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/506>

Post by Tamas Papp
My problem with the key argument is that it complicates the interface. I
would like to use the same interface for sample statistics and random
variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have

If that complicates the interface then don't use them!

&KEY are designed to simplify the interface. If that doesn't work in
your case, don't use it.

Post by Tamas Papp
(mean #(1d0 2d0 3d0)) ; => 2, a sample mean
(mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution
If I had a :KEY argument, I would have to check that it is EQ to
#'identity or not provided in methods for random variables.

Or you can just call the key in all cases.

Post by Tamas Papp
(mean #(1 2 3)) ; => 2
(mean (w/key #'1+ #(1 2 3))) ; => 3

This will never be faster than keys.

Some compilers (such as sbcl), generate different entry points for
functions with &key parameters, so that depending on the lexical keys
specified at the call point, it can jump directly to the right parameter
processing code.

Basically, the compiler behaves like if it generated automatically for:

(defun f (a &key k1 k2) ...)

the following code:

(defun f-0 (a) (let ((k1) (k2)) ...))
(defun f-k1 (a k1) (let ((k2)) ...))
(defun f-k2 (a k2) (let ((k1)) ...))
(defun f-k1-k2 (a k1 k2) ...)
(defun f (a &rest r) (let ((k1 (get-key :k1 r))
(k2 (get-key k2 r)))
...))

;; only smarter, it avoids duplicating the ... code.

and when you call:

(f 42 :k1 11)

it generates:

(f1-k1 42 11)

So what you would do by hand could be done by the compiler.

Only if you called (apply 'f 42 keys) or (f 42 some-key 33)
would it call the complex parameter processing entry point.

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Stas Boukarev

2011-07-04 13:32:36 UTC

Content preview: Nikodemus Siivola writes: > On 4 July 2011 14:46, Stas Boukarev
wrote: > >> * They can't be used with APPLY or FUNCALL. > > Actually, they
can be used with FUNCALL. I meant that it can be use with FUNCALL when it's
called not on a known function. [...]

Content analysis details: (-100.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(stassats[at]gmail.com)
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.214.51 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/499>

Post by Stas Boukarev
* They can't be used with APPLY or FUNCALL.

Actually, they can be used with FUNCALL.

I meant that it can be use with FUNCALL when it's called not on a known
function.

--
With best regards, Stas.

Nathan Froyd

2011-07-04 13:08:59 UTC

Content preview: On Mon, Jul 4, 2011 at 5:57 AM, Tamas Papp wrote: > I understand
this. My main question is: why not do this with compiler > macros? Is there
any reason for this, other than historical? 3.2.2.1.3 might offer some insight:
"However, no language processor (compiler, evaluator, or other code walker)
is ever required to actually invoke compiler macro functions, or to make
use of the resulting expansion if it does invoke a compiler macro function."
[...]

Content analysis details: (-0.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust
[209.85.214.51 listed in list.dnswl.org]
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(froydnj[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/498>

I understand this. My main question is: why not do this with compiler
macros? Is there any reason for this, other than historical?

3.2.2.1.3 might offer some insight: "However, no language processor
(compiler, evaluator, or other code walker) is ever required to
actually invoke compiler macro functions, or to make use of the
resulting expansion if it does invoke a compiler macro function."

-Nathan

Svante Carl v. Erichsen

2011-07-04 10:12:33 UTC

Content preview: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 04.07.2011
11:31, schrieb Tamas Papp: > Why do some CL library functions have :key arguments?

Post by Tamas Papp
I am asking because I am working on some statistics functions, and the

design choice came up. Specifically, I can write functions like > > (defun

quantiles (sequence quantiles &key (key #'identity)) > ...) > > but it is
a bit cumbersome. I can make my code simpler by relying on > calls like >

(quantiles (map 'vector key vector) quantiles) > > but this conses a bit

more. [...]

Content analysis details: (0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(svante.v.erichsen[at]web.de)
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[217.72.192.234 listed in list.dnswl.org]
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
0.0 T_TO_NO_BRKTS_FREEMAIL To: misformatted and free email service
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/489>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the
design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity))
...)
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)
but this conses a bit more.

Doesn't quantiles just pass the key to sort? Is there some smarter
algorithm I am not aware of right now?

Also, passing the key lets you return the complete objects (or
whatever), not just the keys as in the map call. For example, compare

(quantiles #((a 1) (b 4) (c 2) (d 2) (e 5)) 2 :key #'second)

with

(quantiles (map 'vector #'second
#((a 1) (b 4) (c 2) (d 2) (e 5)))
2)

In the first case, you could return (d 2), but not in the second.

Maybe I'm just missing the point. :)

Best wishes,

Svante

- --
Svante Carl v. Erichsen
Wentorfer Str. 96
21029 Hamburg

+49-(0)40-34923721
+49-(0)160-6941474
Svante.v.Erichsen-S0/***@public.gmane.org

Tamas Papp

2011-07-04 10:59:36 UTC

Post by Svante Carl v. Erichsen

Content preview: On Mon, 04 Jul 2011 12:12:33 +0200, Svante Carl v. Erichsen
wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Am 04.07.2011
11:31, schrieb Tamas Papp: >> Why do some CL library functions have :key
arguments? >> >> I am asking because I am working on some statistics functions,
and the >> design choice came up. Specifically, I can write functions like

(defun quantiles (sequence quantiles &key (key #'identity)) >> ...)
but it is a bit cumbersome. I can make my code simpler by relying on

calls like >> >> (quantiles (map 'vector key vector) quantiles) >> >>

but this conses a bit more. > > Doesn't quantiles just pass the key to sort?
Is there some smarter > algorithm I am not aware of right now? [...]

Content analysis details: (-2.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(tkpapp[at]gmail.com)
-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
trust
[80.91.229.12 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 T_TO_NO_BRKTS_FREEMAIL To: misformatted and free email service
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/490>

Post by Svante Carl v. Erichsen
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Doesn't quantiles just pass the key to sort? Is there some smarter
algorithm I am not aware of right now?

Eg

@inproceedings{greenwald2001space,
title={Space-efficient online computation of quantile summaries},
author={Greenwald, M. and Khanna, S.},
booktitle={ACM SIGMOD Record},
volume={30},
number={2},
pages={58--66},
year={2001},
organization={ACM}
}

I am making quantiles a generic function: it should work on objects
returned by the method above, and also on vectors. My problem was that

(quantiles quantile-summary #(0.25 0.5 0.75) :key something)

makes little sense, since the summary is already accumulated. So I will
drop key for the moment, and will probably use compiler macros.

I also thought of a lazy solution that would just save the function and
the object, and traverse once, eg

(quantiles (with-key function object) ...)

but I need to think about that.

Post by Svante Carl v. Erichsen
Also, passing the key lets you return the complete objects (or
whatever), not just the keys as in the map call. For example, compare

That's a good point, I didn't think of that.

Best,

Tamas

Martin Simmons

2011-07-04 11:05:58 UTC

Content preview: >>>>> On Mon, 4 Jul 2011 09:31:31 +0000 (UTC), Tamas Papp
said: > > Why do some CL library functions have :key arguments? > > I am asking
because I am working on some statistics functions, and the > design choice
came up. Specifically, I can write functions like > > (defun quantiles (sequence
quantiles &key (key #'identity)) > ...) > > but it is a bit cumbersome. I
can make my code simpler by relying on > calls like > > (quantiles (map 'vector
key vector) quantiles) [...]

Content analysis details: (-0.0 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
Archived-At: <http://permalink.gmane.org/gmane.lisp.cl-pro/491>

Post by Tamas Papp
Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the
design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity))
...)
but it is a bit cumbersome. I can make my code simpler by relying on
calls like
(quantiles (map 'vector key vector) quantiles)

That approach doesn't work with many of the CL functions (e.g. find, delete,
substitute, remove-duplicates) because they operate on (or return) the
elements of the sequence, not the keys. The :key argument just affects the
arguments to the :test function.

--
Martin Simmons
LispWorks Ltd
http://www.lispworks.com/

Pascal J. Bourguignon

2011-07-04 12:29:19 UTC