[pro] Default safety level and specialized arrays

[Edi Weitz]

Hi everybody,

After rummaging around in the HyperSpec and various implementation
manuals I came to the following conclusion:

1. The standard doesn't prescribe such thing as a default safety level
for CL implementations and there's no portable way to figure out the
safety level of the global environment. (And the same holds for other
optimize qualities.) The closest I could find in the CLHS is 3.3.1
where they talk about an "implementation that always processes code as
if safety were high." From this I conclude that implementations are
allowed NOT to do this or otherwise there would be no need to mention
it.

The LispWorks manual for example describes the defaults in section 9.5
about compiler control, but for SBCL I couldn't find them in the
manual.

2. The standard doesn't prescribe what should happen if you try to
write an element of a wrong type to a specialized array, even in safe
code.

Both LispWorks and SBCL will by default complain in this case but my
understanding is that you can't rely on this in portable code.

So, my question is whether the above is correct or if I missed something?

Thanks,
Edi.

[Pascal J. Bourguignon]

Post by Edi Weitz
Hi everybody,
After rummaging around in the HyperSpec and various implementation
1. The standard doesn't prescribe such thing as a default safety level
for CL implementations and there's no portable way to figure out the
safety level of the global environment. (And the same holds for other
optimize qualities.) The closest I could find in the CLHS is 3.3.1
where they talk about an "implementation that always processes code as
if safety were high." From this I conclude that implementations are
allowed NOT to do this or otherwise there would be no need to mention
it.
The LispWorks manual for example describes the defaults in section 9.5
about compiler control, but for SBCL I couldn't find them in the
manual.
2. The standard doesn't prescribe what should happen if you try to
write an element of a wrong type to a specialized array, even in safe
code.
Both LispWorks and SBCL will by default complain in this case but my
understanding is that you can't rely on this in portable code.
So, my question is whether the above is correct or if I missed something?

Yes.

Notably, with (safety 0), Common Lisp implementation could be as good as common C implementations...

This is why it is very bad to put optimization declarations in libraries (well, if they're not (safety 3) (speed 0)):
it HAS and WILL make users lose days in debugging because YOUR, Edi, libraries, compiled on conforming
implementations fail to detect errors.

[Svante v. Erichsen]

Post by Pascal J. Bourguignon

Post by Edi Weitz
After rummaging around in the HyperSpec and various implementation
1. The standard doesn't prescribe such thing as a default safety level
for CL implementations and there's no portable way to figure out the
safety level of the global environment. (And the same holds for other
optimize qualities.) The closest I could find in the CLHS is 3.3.1
where they talk about an "implementation that always processes code as
if safety were high." From this I conclude that implementations are
allowed NOT to do this or otherwise there would be no need to mention
it.
The LispWorks manual for example describes the defaults in section 9.5
about compiler control, but for SBCL I couldn't find them in the
manual.
2. The standard doesn't prescribe what should happen if you try to
write an element of a wrong type to a specialized array, even in safe
code.
Both LispWorks and SBCL will by default complain in this case but my
understanding is that you can't rely on this in portable code.
So, my question is whether the above is correct or if I missed something?

Yes.
Notably, with (safety 0), Common Lisp implementation could be as good as common C implementations...
it HAS and WILL make users lose days in debugging because YOUR, Edi, libraries, compiled on conforming
implementations fail to detect errors.

Hello!

This begs the question: how do I get a speed optimized library then? Do I need
to make a local fork just to insert declarations? Is there a way to tell
e. g. ASDF to use certain declamations? Is making the declarations in the
library dependent on documented entries to *features* the way to go? Where
would I push the desired settings into *features*?

Yours aye

Svante

[Pascal J. Bourguignon]

Post by Svante v. Erichsen
This begs the question: how do I get a speed optimized library then? Do I need
to make a local fork just to insert declarations? Is there a way to tell

This is not up to the library writer to decde on this. This is up to the library final user.

Declarations are global. When you are developping/debugging, you will set them to
(optimize (speed 0) (space 0) (debug 3) (safety 3)).

When generating the end-user executable, you will set them to
(optimize (speed 3) (space 3) (debug 0) (safety 3)),
if you follow Bertrand Meyer's advice.

Post by Svante v. Erichsen
e. g. ASDF to use certain declamations? Is making the declarations in the
library dependent on documented entries to *features* the way to go? Where
would I push the desired settings into *features*?

Even if you set them in ASDF, you will have to have a way to tell it when you are debugging code (always), and when you are generating a end-user application (once in a blue moon, and in a different, explicitely controlled process).

And yes, since asdf doesn't cache the compiled files in separate directories for the various optimization settings (or the various *features* list), since that would be very unwiedly, you have to clear the cache when you switch between generating the end-user application, and normal developping work. rm -rf ~/.cache/common-lisp/ is found in all application generation makefiles.

Optimize declaimations and declarations in libraries is the curse of the CL application developper.

[Ralf Mattes]

Post by Pascal J. Bourguignon
And yes, since asdf doesn't cache the compiled files in separate directories for the various optimization
settings (or the various *features* list), since that would be very unwiedly, you have to clear the cache
when you switch between generating the end-user application, and normal developping work.
rm -rf ~/.cache/common-lisp/ is found in all application generation makefiles.

Hmm, I think ASDF_OUTPUT_TRANSLATIONS still works, so my makefile looks like:

LISP := "/usr/local/bin/sbcl"
ARGS := "--non-interactive"

debug: export ASDF_OUTPUT_TRANSLATIONS = "/:/tmp/lisp-cache/$@/"
debug:
$(LISP) $(ARGS) --load $@.lisp --load build.lisp

release: export ASDF_OUTPUT_TRANSLATIONS = "/:/tmp/lisp-cache/$@/"
release:
$(LISP) $(ARGS) --load $@.lisp --load build.lisp

where debug.lisp / release.lisp would contain the wanted optimization settings.

HTH Ralf Mattes

[Faré]

Post by Pascal J. Bourguignon

Post by Svante v. Erichsen
This begs the question: how do I get a speed optimized library then?
Do I need to make a local fork just to insert declarations?
Is there a way to tell

This is not up to the library writer to decde on this. This is up to the
library final user.
Declarations are global. When you are developping/debugging, you will set them to
(optimize (speed 0) (space 0) (debug 3) (safety 3)).
When generating the end-user executable, you will set them to
(optimize (speed 3) (space 3) (debug 0) (safety 3)),
if you follow Bertrand Meyer's advice.

Post by Svante v. Erichsen
e. g. ASDF to use certain declamations? Is making the declarations in the
library dependent on documented entries to *features* the way to go? Where
would I push the desired settings into *features*?

Even if you set them in ASDF, you will have to have a way to tell it when
you are debugging code (always), and when you are generating a end-user
application (once in a blue moon, and in a different, explicitely controlled
process).
And yes, since asdf doesn't cache the compiled files in separate directories
for the various optimization settings (or the various *features* list),
since that would be very unwiedly, you have to clear the cache when you
switch between generating the end-user application, and normal developping
work. rm -rf ~/.cache/common-lisp/ is found in all application generation
makefiles.
Optimize declaimations and declarations in libraries is the curse of the CL
application developper.

For an involved way of solving this problem as part of a reproducible build
of a company's software, see how we do it in the quux snapshot:
https://common-lisp.net/project/qitab/
http://common-lisp.net/project/qitab/archives/quux-2013-09-24.tar.bz2

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
The [classical] liberal, of course, does not deny that there are
some superior people — he is not an egalitarian — but he denies
that anyone has authority to decide who these superior people are.
— F. A. Hayek, "Why I Am Not a Conservative"