Proper behavior of slot-initforms in defstruct?

[Jean-Claude Beaudoin]

Please consider the following code:

(defparameter init-a 1)

(let ((init-a 42) (serial-no 0))
(defstruct foo (a init-a) (b (incf serial-no)))
(defun get-foo-serial-no () serial-no))

(defstruct (bar (:include foo)) (c 33) d)

When one loads the above and then try to call #'make-bar the result
varies widely from one lisp implementation (clisp) to another.

clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL)
ccl: (make-bar) --> <enter the debugger saying: "Unbound variable:
SERIAL-NO">

lispworks, allegro and sbcl also behave more or less like ccl.

What is the proper ANSI-CL behavior in this case here?
Is clisp right in evaluating the slot initform in its "proper" lexical
context?
Or is the correct behavior to replicate the slot initform verbatim
in the sub-structure constructor regardless of its original lexical context
like the others do?

I guess that this question has probably been asked before, in a somewhat
distant past, but my google skills have not been sharp enough to find it,
sorry.

[Laughing Water]

It looks like you need a LET* to guarantee the order of evaluation within your LET. Otherwise, it’s undefined.

Laughing Water

Post by Jean-Claude Beaudoin
(defparameter init-a 1)
(let ((init-a 42) (serial-no 0))
(defstruct foo (a init-a) (b (incf serial-no)))
(defun get-foo-serial-no () serial-no))
(defstruct (bar (:include foo)) (c 33) d)
When one loads the above and then try to call #'make-bar the result
varies widely from one lisp implementation (clisp) to another.
clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL)
ccl: (make-bar) --> <enter the debugger saying: "Unbound variable: SERIAL-NO">
lispworks, allegro and sbcl also behave more or less like ccl.
What is the proper ANSI-CL behavior in this case here?
Is clisp right in evaluating the slot initform in its "proper" lexical context?
Or is the correct behavior to replicate the slot initform verbatim
in the sub-structure constructor regardless of its original lexical context
like the others do?
I guess that this question has probably been asked before, in a somewhat
distant past, but my google skills have not been sharp enough to find it, sorry.

[Peter Stirling]

That makes no sense, the bindings in the let are independent.

Post by Laughing Water
It looks like you need a LET* to guarantee the order of evaluation within your LET. Otherwise, it’s undefined.
Laughing Water

Post by Jean-Claude Beaudoin
(defparameter init-a 1)
(let ((init-a 42) (serial-no 0))
(defstruct foo (a init-a) (b (incf serial-no)))
(defun get-foo-serial-no () serial-no))
(defstruct (bar (:include foo)) (c 33) d)
When one loads the above and then try to call #'make-bar the result
varies widely from one lisp implementation (clisp) to another.
clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL)
ccl: (make-bar) --> <enter the debugger saying: "Unbound variable: SERIAL-NO">
lispworks, allegro and sbcl also behave more or less like ccl.
What is the proper ANSI-CL behavior in this case here?
Is clisp right in evaluating the slot initform in its "proper" lexical context?
Or is the correct behavior to replicate the slot initform verbatim
in the sub-structure constructor regardless of its original lexical context
like the others do?
I guess that this question has probably been asked before, in a somewhat
distant past, but my google skills have not been sharp enough to find it, sorry.

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[Peter Stirling]

My read of the spec is that either behaviour is allowable.

DEFSTRUCT[1] says:

"If a slot is not initialized in this way, it is initialized by
evaluating /slot-initform/ in the slot description at the time the
constructor function is called."

Which makes no reference to the environment in which the form should be
evaluated in.

Later on it says:

"It is as if the /slot-initforms/ were used as /initialization
forms/
<http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_i.htm#initialization_form>
for the /keyword parameters/
<http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_k.htm#keyword_parameter>
of the constructor function"

The page describing keyword arguments in ordinary lambda lists[2] also
makes no reference to which environment should be used for initforms.

I guess they didn't think about people using closures for initforms when
they were drawing it up?

To side-step the issue I would invoke a closure from the initform
instead (if that's how you want to do it).

[1] http://www.lispworks.com/documentation/HyperSpec/Body/m_defstr.htm
[2] http://www.lispworks.com/documentation/HyperSpec/Body/03_dad.htm

Post by Peter Stirling
That makes no sense, the bindings in the let are independent.

Post by Laughing Water
It looks like you need a LET* to guarantee the order of evaluation
within your LET. Otherwise, it’s undefined.
Laughing Water

On Aug 3, 2015, at 3:27 PM, Jean-Claude Beaudoin
(defparameter init-a 1)
(let ((init-a 42) (serial-no 0))
(defstruct foo (a init-a) (b (incf serial-no)))
(defun get-foo-serial-no () serial-no))
(defstruct (bar (:include foo)) (c 33) d)
When one loads the above and then try to call #'make-bar the result
varies widely from one lisp implementation (clisp) to another.
clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL)
ccl: (make-bar) --> <enter the debugger saying: "Unbound variable: SERIAL-NO">
lispworks, allegro and sbcl also behave more or less like ccl.
What is the proper ANSI-CL behavior in this case here?
Is clisp right in evaluating the slot initform in its "proper" lexical context?
Or is the correct behavior to replicate the slot initform verbatim
in the sub-structure constructor regardless of its original lexical context
like the others do?
I guess that this question has probably been asked before, in a somewhat
distant past, but my google skills have not been sharp enough to find it, sorry.

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[Jean-Claude Beaudoin]

Post by Peter Stirling
My read of the spec is that either behaviour is allowable.

The two behaviors are so different that I have a hard time accepting this.

Post by Peter Stirling
I guess they didn't think about people using closures for initforms when
they were drawing it up?

Yet they clearly thought about closures when they specified DEFCLASS[3]
(see the part where
processing of :initform form is mentioned). So, was this really just simple
oversight?

To side-step the issue I would invoke a closure from the initform instead

Post by Peter Stirling
(if that's how you want to do it).

This is indeed probably the proper workaround that would assure portability
across CL implementation.

But I find sad that we'd have to force defstruct to be a top-level only form
with this workaround instead of letting perfectly normal language
constructs intermix
freely as the problem would find appropriate and natural.

My interest here is that I am reworking the implementation of the DEFSTRUCT
macro in MKCL.
And honestly I am of the opinion that there is a pretty strong case to be
made in favor
of the clisp behavior on this issue. But I cannot figure out the
justification for the other
behavior illustrated by almost all of the other implementations. Is this
just some long
standing historical quirk at play here or is there really a reason for this
way of doing it?


[3] http://www.lispworks.com/documentation/HyperSpec/Body/m_defcla.htm

[Kenneth Tilton]

On Tue, Aug 4, 2015 at 3:55 AM, Jean-Claude Beaudoin <

Post by Jean-Claude Beaudoin

Post by Peter Stirling
My read of the spec is that either behaviour is allowable.

The two behaviors are so different that I have a hard time accepting this.

The issue is not the degree of behavior difference, the issue is the degree
to which the language of the spec constrains an implementor.

Did Peter miss this, or am I missing something completely (I am just a
simple application programmer): "The slot default init forms are evaluated
in the lexical environment in which the defstruct form itself appears and
in the dynamic environment in which the call to the constructor function
appears.

-kt
--
Kenneth Tilton
54 Isle of Venice Dr
Fort Lauderdale, FL 33301

***@tiltontec.com
http://tiltontec.com
@tiltonsalgebra

646-269-1077

"In a class by itself." *-Macworld*

[Jean-Claude Beaudoin]

Post by Kenneth Tilton
On Tue, Aug 4, 2015 at 3:55 AM, Jean-Claude Beaudoin <

On Mon, Aug 3, 2015 at 10:12 PM, Peter Stirling <

Post by Peter Stirling
My read of the spec is that either behaviour is allowable.

The two behaviors are so different that I have a hard time accepting this.

The issue is not the degree of behavior difference, the issue is the
degree to which the language of the spec constrains an implementor.
Did Peter miss this, or am I missing something completely (I am just a
simple application programmer): "The slot default init forms are evaluated
in the lexical environment in which the defstruct form itself appears and
in the dynamic environment in which the call to the constructor function
appears.

It seems that Peter and I both missed it somehow. That clears the issue
pretty clearly;
clisp wins and all the others, well...

Thank you very much Ken for pointing this key sentence.

Case closed.

[Kenneth Tilton]

On Tue, Aug 4, 2015 at 5:01 AM, Jean-Claude Beaudoin <

Post by Jean-Claude Beaudoin

Post by Kenneth Tilton
On Tue, Aug 4, 2015 at 3:55 AM, Jean-Claude Beaudoin <

On Mon, Aug 3, 2015 at 10:12 PM, Peter Stirling <

Post by Peter Stirling
My read of the spec is that either behaviour is allowable.

The two behaviors are so different that I have a hard time accepting this.

The issue is not the degree of behavior difference, the issue is the
degree to which the language of the spec constrains an implementor.
Did Peter miss this, or am I missing something completely (I am just a
simple application programmer): "The slot default init forms are evaluated
in the lexical environment in which the defstruct form itself appears and
in the dynamic environment in which the call to the constructor function
appears.

It seems that Peter and I both missed it somehow.

I missed it, too, the first 2-3 times. Then I happened to look back at the
section and there it was as plain as day. Perception is like that.

Post by Jean-Claude Beaudoin
That clears the issue pretty clearly;
clisp wins and all the others, well...

No, CLisp loses (a little*).

In the dynamic environment of the execution of your top-level (make-bar)
example the variable serial-no is unbound, so evaluating the initform (incf
serial-no) should fail as you documented.

The CLisp source is available, AFAIK. I suspect we would discover that the
initform is implemented as an anonymous function that closes over the
lexically available serial-no at definition time, and that the :include
option does not copy the slot *definition* and then proceed with struct
compilation*, instead it copies the compilation* and hence the same closure
over the same serial-no.

* Probably relevant: Clisp is interpreted.

-hk

Post by Jean-Claude Beaudoin
Thank you very much Ken for pointing this key sentence.
Case closed.

--
Kenneth Tilton
54 Isle of Venice Dr
Fort Lauderdale, FL 33301

***@tiltontec.com
http://tiltontec.com
@tiltonsalgebra

646-269-1077

"In a class by itself." *-Macworld*

[Martin Simmons]

Post by Jean-Claude Beaudoin
But I find sad that we'd have to force defstruct to be a top-level only form
with this workaround instead of letting perfectly normal language constructs
intermix freely as the problem would find appropriate and natural.
My interest here is that I am reworking the implementation of the DEFSTRUCT
macro in MKCL. And honestly I am of the opinion that there is a pretty
strong case to be made in favor of the clisp behavior on this issue. But I
cannot figure out the justification for the other behavior illustrated by
almost all of the other implementations. Is this just some long standing
historical quirk at play here or is there really a reason for this way of
doing it?

Probably because remembering the initform and splicing it into the constructor
of the included defstruct is the most obvious implementation.

It is easy to make that implementation evaluate the initform in the correct
lexical environment by including the constructor in the macroexpansion of
defstruct. Doing that for included defstructs requires more effort to avoid
slowdown in the simple cases.

Another related case is EQ'ness of the initform values. For example, with two
files:

q1.lisp:
(defstruct q1 (a '(1 2 3)))
(defun t1 () (eq (q1-a (make-q1)) (q1-a (make-q1))))

q2.lisp:
(defstruct (q2 (:include q1)))
(defun t12 () (eq (q1-a (make-q1)) (q1-a (make-q2))))

If you compile and load each of these files, then t1 should always return t
whereas t12 might return nil or t depending on whether the code for the
initform in q1 is shared or copied.

[Kenneth Tilton]

Post by Peter Stirling
My read of the spec is that either behaviour is allowable.
"If a slot is not initialized in this way, it is initialized by
evaluating *slot-initform* in the slot description at the time the
constructor function is called."
Which makes no reference to the environment in which the form should be
evaluated in.
"It is as if the *slot-initforms* were used as *initialization forms*
<http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_i.htm#initialization_form>
for the *keyword parameters*
<http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_k.htm#keyword_parameter>
of the constructor function"

Later later on it says:

"The slot default init forms are evaluated in the lexical environment in
which the defstruct form itself appears and in the dynamic environment in
which the call to the constructor function appears."

So the defstruct sees the local/lexical serial-no but a call to make-bar
outside does not.

In between we get more clues: "The symbols which name the slots must not be
used by the implementation as the names for the lambda variables in the
constructor function, since one or more of thosesymbols might have been
proclaimed special or might be defined as the name of a constant variable. "

Indeed, if one compiles the example provided, when one *compiles *(ACL has
both an interpreter and compiler, and they vary) make-bar one learns:

Warning: Free reference to undeclared variable serial-no assumed special.

I checked the spec on included structs but did not notice anything about
included slot init-form evaluation, but I guess it makes sense that
included slots be assembled and evaluated as if coded with the including
struct.

-hk

Post by Peter Stirling
The page describing keyword arguments in ordinary lambda lists[2] also
makes no reference to which environment should be used for initforms.
I guess they didn't think about people using closures for initforms when
they were drawing it up?
To side-step the issue I would invoke a closure from the initform instead
(if that's how you want to do it).
[1] http://www.lispworks.com/documentation/HyperSpec/Body/m_defstr.htm
[2] http://www.lispworks.com/documentation/HyperSpec/Body/03_dad.htm
That makes no sense, the bindings in the let are independent.
It looks like you need a LET* to guarantee the order of evaluation within
your LET. Otherwise, it’s undefined.
Laughing Water
On Aug 3, 2015, at 3:27 PM, Jean-Claude Beaudoin
(defparameter init-a 1)
(let ((init-a 42) (serial-no 0))
(defstruct foo (a init-a) (b (incf serial-no)))
(defun get-foo-serial-no () serial-no))
(defstruct (bar (:include foo)) (c 33) d)
When one loads the above and then try to call #'make-bar the result
varies widely from one lisp implementation (clisp) to another.
clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL)
ccl: (make-bar) --> <enter the debugger saying: "Unbound variable: SERIAL-NO">
lispworks, allegro and sbcl also behave more or less like ccl.
What is the proper ANSI-CL behavior in this case here?
Is clisp right in evaluating the slot initform in its "proper" lexical context?
Or is the correct behavior to replicate the slot initform verbatim
in the sub-structure constructor regardless of its original lexical context
like the others do?
I guess that this question has probably been asked before, in a somewhat
distant past, but my google skills have not been sharp enough to find it, sorry.
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

--
Kenneth Tilton
54 Isle of Venice Dr
Fort Lauderdale, FL 33301

***@tiltontec.com
http://tiltontec.com
@tiltonsalgebra

646-269-1077

"In a class by itself." *-Macworld*