Common Lisp style: multiple packages in same repo

[Daniel Pezely]

I posted this to Stack Overflow but was hoping for more input.

https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo


May I get recommendations or links to representative code repositories
with good style for multiple related Common Lisp packages, please?

For instance, consider a high-level workflow library with accompanying
lower-level API, each in its own CL package but same git repo due to
synchronized releases.

Each system (|*.asd| file) isolates tests and may be invoked using:

|(asdf:test-system foo :force t)|

Separate systems may be built via |make|, which definitely helps isolate
SBCL code-coverage reports.

Some users of the library may only want to load the lower-level API. For
simplifying dependencies for those using higher-level API, it seems best
to keep everything bundled in one repo. Any revision to one library
would likely require updating all for the same release.

I currently have a single directory tree with a subdirectory for each CL
package. There's a top-level |Makefile| plus one in each subdirectory
that the library maintain would use. The top-level also contains
symbolic links for |.asd| files pointing into relevant subdirectories.
(It's a library deeply dependent upon POSIX calls via |uiop-posix|, so
it's only applicable on an OS with sym-links.)

This seems to be an issue at large considering issue #1 for
Quicklisp-docs [0].

Found nothing relevant in Google's CL style guide [1], State of the
Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4].
Browsing repos seem to have quite a mix of styles.

* [0] https://github.com/rudolfochrist/quicklisp-docs/issues/1
* [1] https://google.github.io/styleguide/lispguide.xml
* [2]
https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/
* [3] http://weitz.de/cl-recipes/
* [4] http://lisp-lang.org/learn/writing-libraries but see also their
section on Continuous Integration

Repo to be fixed: https://gitlab.com/dpezely/cl-mmap
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)

Thanks,
  -Daniel

[Pascal Bourguignon]

Post by Daniel Pezely
I posted this to Stack Overflow but was hoping for more input.
https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo <https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo>
May I get recommendations or links to representative code repositories with good style for multiple related Common Lisp packages, please?
For instance, consider a high-level workflow library with accompanying lower-level API, each in its own CL package but same git repo due to synchronized releases.
(asdf:test-system foo :force t)
Separate systems may be built via make, which definitely helps isolate SBCL code-coverage reports.
Some users of the library may only want to load the lower-level API. For simplifying dependencies for those using higher-level API, it seems best to keep everything bundled in one repo. Any revision to one library would likely require updating all for the same release.
I currently have a single directory tree with a subdirectory for each CL package. There's a top-level Makefile plus one in each subdirectory that the library maintain would use. The top-level also contains symbolic links for .asd files pointing into relevant subdirectories. (It's a library deeply dependent upon POSIX calls via uiop-posix, so it's only applicable on an OS with sym-links.)
This seems to be an issue at large considering issue #1 for Quicklisp-docs [0].
Found nothing relevant in Google's CL style guide [1], State of the Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing repos seem to have quite a mix of styles.
[0] https://github.com/rudolfochrist/quicklisp-docs/issues/1 <https://github.com/rudolfochrist/quicklisp-docs/issues/1>
[1] https://google.github.io/styleguide/lispguide.xml <https://google.github.io/styleguide/lispguide.xml>
[2] https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/ <https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/>
[3] http://weitz.de/cl-recipes/ <http://weitz.de/cl-recipes/>
[4] http://lisp-lang.org/learn/writing-libraries <http://lisp-lang.org/learn/writing-libraries> but see also their section on Continuous Integration
Repo to be fixed: https://gitlab.com/dpezely/cl-mmap <https://gitlab.com/dpezely/cl-mmap>
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)
Thanks,
-Daniel

You’re asking about different things.

- git repositories. It’s more a question of ownership and administration than anything else. Technically, as you noted, it is better to keep in the same repository things that will evolve in sync. eg. client & server module implementing the same protocol. But if for some reason those two modules had to be developed by different teams, it would still be preferable to have two separate repositories, and synchronize thru the protocol specification, and not thru the code. Otherwise, as a small team of 1 I tend to prefer big git repositories holding everything eg. http://github.com/informatimago/lisp

- asdf systems are entirely orthogonal and agnostic to packages. It will let you do as you wish. Whether what you do is good or not will depend on the use case.
It may be the less surprising option to map 1 package to 1 system, and to avoid defining (or overriding/redefining) things in other packages. But it may be needed, useful or easier to ignore this rule.

For example, you can define a package X loaded with a system X. Then a package X.TEST loaded with a system X.TEST. But in order to test some internal feature of X, you may have to define or redefine things in the package X, from the code loaded by the system X.TEST; or you may want to avoid the package X.TEST altogether, and just load tests in the package X, from the system named X.TEST! All the other use cases are possible, and might be justified by a given situation.

- files. Mapping lisp definitions (and other top-level forms) to files is rather arbitrary. The only technical constraint is that the definitions that are used by macros (at macroexpansion-time ∈ compilation-time), must be compiled before, and loaded into the compilation environment before those macros can be used (and even, better before they’re defined). This can be done by wrapping those definitions in an eval-when form, but it’s often easier to put them in a separate file, to be compiled and loaded before the file defining the macro and the files using it. For the rest, do as you wish. I try to follow the logic of the program, or the “explanatory order” (think about the human reader). But it doesn’t matter at all. Instead of worrying about that, you’d better spend your time writing an emacs mode that would store lisp forms into files automatically. You’d just have a browser (like a smalltalk browser), and add definitions, or browse and select definitions to be edited. emacs would fetch them from the file IT had decided where to store them, and would present them in an ephemeral buffer for your edit. Of course, instead of text-based commands such as search and replace, you’d have symbolic editing commands such as rename-symbol-in-definition or rename-symbol-everywhere or substitute-sexp-everywhere etc. This includes directories.
That said, nothing prevents you to define some conventions for your current project. See again the above github repo for an example. I’m not saying you should follow it, it’s just a random example that suits me for this case, an aggregation of libraries. For application programs, I’d use a totally different way to structure code in packages, and spread code in files and directories, and how to load it with what systems.

Basically, what I’m saying is that it’s a software engineering question and it must be answered based on software engineering design and decisions. In UML you have the concepts of module and components, and with stereotypes, different kinds of components, you can represent packages, asdf system and components, and even if you want to modelize it, asdf files, since asdf uses files as unit of compilation and loading. How you structure your software cannot be PRE-designed! You have to do the work yourself, as a software engineer!

In short: it depends!

[Daniel Pezely]

Post by Pascal Bourguignon

Post by Daniel Pezely
I posted this to Stack Overflow but was hoping for more input.
https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo
May I get recommendations or links to representative code
repositories with good style for multiple related Common Lisp
packages, please? [...]

You’re asking about different things.
[...]
In short: it depends!

Thanks, Pascal!

Yes, these are different-- yet intertwined as integral parts of a whole.

Regarding git repos: in this case, it's just one person.  The source
code is small enough that it would be maintained by one person for
foreseeable future, including potential forks supplying missing features
that might be apparent to those familiar with conventional mmap use cases.

Regarding asdf systems versus packages: the hooks for invoking the test
harness (Prove), launches via asdf mechanisms.  For practical purposes,
these are tightly coupled here.

Regarding files: as much as I admire the Smalltalk model, my pragmatic
side tips the scales in favour of being confined to conventional CL
tools and approach, hence: asdf, uiop-posix, Prove and accommodating
possible future inclusion by Quicklisp.


Oddly enough, I was expecting to drink from a fire hose of highly
opinionated yet qualified CL advice.

By way of comparison, I just returned from Rust Conf in Portland,
Oregon/US, last week and was reminded that their `cargo` package manager
has "workspaces" [1].  It synchronizes dependencies of multiple packages
within a common directory tree.  That satisfies criteria described in
the original post, apart from being for a different language of course.

(Unfortunately, I don't have time to contribute a Lispy equivalent as
the most recent start-up company didn't quite go as intended... but I
digress.)

[1]
https://doc.rust-lang.org/book/second-edition/ch14-03-cargo-workspaces.html


Thanks again for all your recommendations and insights!

  -Daniel

[Ken Tilton]

Packages are massively overrated. This is not Java where every frickin
source file is a namespace. There is a certain obsessive compulsiveness
about packages that does nothing but slow developers down. Well, right,
they are a palliative for the OCD disease. But it *is* a disease, so that
does not count.

What part of agile do we not understand? Fences, boxes, categories, types
all invented for their own sake let us bask in our taxonomicity while
getting no code written, and god help the sucker who tries to use our OCD
mess forever battling package issues.

Stop. Wrong way. Go back.

Post by Daniel Pezely
I posted this to Stack Overflow but was hoping for more input.
https://stackoverflow.com/questions/51638864/common-
lisp-style-multiple-packages-in-same-repo
May I get recommendations or links to representative code repositories
with good style for multiple related Common Lisp packages, please?
For instance, consider a high-level workflow library with accompanying
lower-level API, each in its own CL package but same git repo due to
synchronized releases.
(asdf:test-system foo :force t)
Separate systems may be built via make, which definitely helps isolate
SBCL code-coverage reports.
Some users of the library may only want to load the lower-level API. For
simplifying dependencies for those using higher-level API, it seems best to
keep everything bundled in one repo. Any revision to one library would
likely require updating all for the same release.
I currently have a single directory tree with a subdirectory for each CL
package. There's a top-level Makefile plus one in each subdirectory that
the library maintain would use. The top-level also contains symbolic links
for .asd files pointing into relevant subdirectories. (It's a library
deeply dependent upon POSIX calls via uiop-posix, so it's only applicable
on an OS with sym-links.)
This seems to be an issue at large considering issue #1 for Quicklisp-docs
[0].
Found nothing relevant in Google's CL style guide [1], State of the Common
Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing
repos seem to have quite a mix of styles.
- [0] https://github.com/rudolfochrist/quicklisp-docs/issues/1
- [1] https://google.github.io/styleguide/lispguide.xml
- [2] https://web.archive.org/web/20160305061250/http://eudoxia.
me/article/common-lisp-sotu-2015/
<https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/>
- [3] http://weitz.de/cl-recipes/
- [4] http://lisp-lang.org/learn/writing-libraries but see also their
section on Continuous Integration
Repo to be fixed: https://gitlab.com/dpezely/cl-mmap
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)
Thanks,
-Daniel

[Scott McKay]

+1

I think we did OK in CLIM: an API package, and an internal package to implement in.

--Scott

Packages are massively overrated. This is not Java where every frickin source file is a namespace. There is a certain obsessive compulsiveness about packages that does nothing but slow developers down. Well, right, they are a palliative for the OCD disease. But it *is* a disease, so that does not count.
What part of agile do we not understand? Fences, boxes, categories, types all invented for their own sake let us bask in our taxonomicity while getting no code written, and god help the sucker who tries to use our OCD mess forever battling package issues.
Stop. Wrong way. Go back.

Post by Daniel Pezely
I posted this to Stack Overflow but was hoping for more input.
https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo
May I get recommendations or links to representative code repositories with good style for multiple related Common Lisp packages, please?
For instance, consider a high-level workflow library with accompanying lower-level API, each in its own CL package but same git repo due to synchronized releases.
(asdf:test-system foo :force t)
Separate systems may be built via make, which definitely helps isolate SBCL code-coverage reports.
Some users of the library may only want to load the lower-level API. For simplifying dependencies for those using higher-level API, it seems best to keep everything bundled in one repo. Any revision to one library would likely require updating all for the same release.
I currently have a single directory tree with a subdirectory for each CL package. There's a top-level Makefile plus one in each subdirectory that the library maintain would use. The top-level also contains symbolic links for .asd files pointing into relevant subdirectories. (It's a library deeply dependent upon POSIX calls via uiop-posix, so it's only applicable on an OS with sym-links.)
This seems to be an issue at large considering issue #1 for Quicklisp-docs [0].
Found nothing relevant in Google's CL style guide [1], State of the Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing repos seem to have quite a mix of styles.
[0] https://github.com/rudolfochrist/quicklisp-docs/issues/1
[1] https://google.github.io/styleguide/lispguide.xml
[2] https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/
[3] http://weitz.de/cl-recipes/
[4] http://lisp-lang.org/learn/writing-libraries but see also their section on Continuous Integration
Repo to be fixed: https://gitlab.com/dpezely/cl-mmap
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)
Thanks,
-Daniel

--
Kenneth Tilton
http://tiltontec.com/

[Faré]

I've seen flat namespaces and one-package-per-file and everything
in-between and beyond. All style can be done well or be done poorly.
What matters most is that namespaces must be curated to be livable.
Having someone responsible to maintain it, make it good, make it
consistent, keep it good, fix the bugs, improve the interfaces, etc. —
That's the important and hard part.

That said, I admit I've grown to like the approach of ASDF and LIL, of
one-package-per-file internally (also each file a system, as for LIL),
plus special package files that (automatically) import and reexport
all symbols in a directory. This style imposes a somewhat onerous
discipline, but in exchange you have a powerful framework to keep you
code well-organized: each file by construction can only refer to
symbols from files that it depends on, you have think of your
dependencies clearly. Any forward reference requires special
provisions. Without this discipline, ASDF could not have grown up from
the 4516 line mess that ASDF had become as of 2.6, to the
well-organized 13525 line robust and portable software that it is as
of 3.3.2.9. It does so much more under the hood, and is portable to so
many more platforms, yet at the same time the code is so much more
readable, and the underlying algorithm, while much more elaborate, is
also much more understandable. One of the few things that survives of
my work at ITA :-(

As for LIL, it also uses packages for punning: there are distinct but
related symbols stateful:insert pure:insert classy:insert and
posh:insert for inserting a key / value pair in a map in all
combinations of stateful interface-passing-style, pure
interface-passing-style, classic imperative object oriented style, and
purely functional object oriented style — with macros that can
automatically wrap one into the others. That's incredibly powerful and
liberating: being able to use parametric polymorphism to bootstrap
elaborate pure functional data structures like Haskell typeclasses,
then exporting classic imperative object-oriented hierarchies for
users who don't want to know.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Trillions of fossils can't be wrong!

Post by Scott McKay
+1
I think we did OK in CLIM: an API package, and an internal package to implement in.
--Scott
Packages are massively overrated. This is not Java where every frickin source file is a namespace. There is a certain obsessive compulsiveness about packages that does nothing but slow developers down. Well, right, they are a palliative for the OCD disease. But it *is* a disease, so that does not count.
What part of agile do we not understand? Fences, boxes, categories, types all invented for their own sake let us bask in our taxonomicity while getting no code written, and god help the sucker who tries to use our OCD mess forever battling package issues.
Stop. Wrong way. Go back.

Post by Daniel Pezely
I posted this to Stack Overflow but was hoping for more input.
https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-packages-in-same-repo
May I get recommendations or links to representative code repositories with good style for multiple related Common Lisp packages, please?
For instance, consider a high-level workflow library with accompanying lower-level API, each in its own CL package but same git repo due to synchronized releases.
(asdf:test-system foo :force t)
Separate systems may be built via make, which definitely helps isolate SBCL code-coverage reports.
Some users of the library may only want to load the lower-level API. For simplifying dependencies for those using higher-level API, it seems best to keep everything bundled in one repo. Any revision to one library would likely require updating all for the same release.
I currently have a single directory tree with a subdirectory for each CL package. There's a top-level Makefile plus one in each subdirectory that the library maintain would use. The top-level also contains symbolic links for .asd files pointing into relevant subdirectories. (It's a library deeply dependent upon POSIX calls via uiop-posix, so it's only applicable on an OS with sym-links.)
This seems to be an issue at large considering issue #1 for Quicklisp-docs [0].
Found nothing relevant in Google's CL style guide [1], State of the Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing repos seem to have quite a mix of styles.
[0] https://github.com/rudolfochrist/quicklisp-docs/issues/1
[1] https://google.github.io/styleguide/lispguide.xml
[2] https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/
[3] http://weitz.de/cl-recipes/
[4] http://lisp-lang.org/learn/writing-libraries but see also their section on Continuous Integration
Repo to be fixed: https://gitlab.com/dpezely/cl-mmap
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)
Thanks,
-Daniel

--
Kenneth Tilton
http://tiltontec.com/

[Daniel Pezely]

[...] keep your
code well-organized: each file by construction can only refer to
symbols from files that it depends on, you have think of your
dependencies clearly. [...]

I generally strive for a similar discipline but on the package level
rather than granularity of individual files.

An observation supporting these approaches: Use of fully qualified
symbol names makes it easier for other people to navigate and reason
able unfamiliar code.  Package nicknames in CL help with otherwise
cumbersome names, so there's little reason not to do this in my experience.

As for LIL, it also uses packages for punning: there are distinct but
related symbols stateful:insert pure:insert classy:insert and
posh:insert for inserting a key / value pair in a map in all
combinations of stateful interface-passing-style, pure
interface-passing-style, classic imperative object oriented style, and
purely functional object oriented style — with macros that can
automatically wrap one into the others. [...]

Yes, for me, something similar justified two systems, two packages in
one repo.  The higher level package shadows some CL names like OPEN for
capitalizing upon one's existing knowledge of CL:OPEN's parameters and
behaviours.

Understandably, some people avoid shadowing of CL functions, so this
contributed to splitting into two packages-- beyond higher versus lower
level interfaces.


Faré, you bring up many interesting points for possible further
discussion, and thank you for mentioning LIL.  I had forgotten about LIL
long since glancing at ILC'12 proceedings all those years ago. I'll
definitely explore techniques you mentioned for other packages I intend
to release in future.


Regarding questions from the original post, the Common Lisp community at
large seems far from attaining a quorum, let alone consensus.

Perhaps this presents an opportunity for maintainers of Quicklisp
(perhaps in collaboration with Quickdocs) to introduce a style guide for
making their lives easier.  Maybe after gaining more experience with
Rust's (Cargo's) workspaces [1] for synchronizing versions across a
family of subsystems, I'll propose something to Quicklisp maintainers. 
I'm sure Rust didn't invent that mechanism, so if others possess that
knowledge already, please suggest such a model!

[1]
https://github.com/rust-lang/rfcs/blob/master/text/1525-cargo-workspace.md

Thanks,
  -Daniel

[Faré]

Post by Daniel Pezely
Perhaps this presents an opportunity for maintainers of Quicklisp
(perhaps in collaboration with Quickdocs) to introduce a style guide for
making their lives easier.

I edited the Google Common Lisp Style Guide (well, at least the
version that was published a few years back; I know it has evolved
internally since, but no updates were published.) I also published an
ASDF "best practices" document.

There are many acceptable yet mutually incompatible ways of using
Common Lisp packages. You must pick one and stick to it within any
given system. Maybe you can describe them (or at least, one) in a blog
post and/or some other document that you write.

Then again, there are also proposed alternatives to packages. Or you
may embrace modernity and embrace Racket modules, or their
reimplementation by Gerbil Scheme.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
It is paradoxical, yet true, to say, that the more we know, the more ignorant
we become in the absolute sense, for it is only through enlightenment that we
become conscious of our limitations. Precisely one of the most gratifying
results of intellectual evolution is the continuous opening up of new and
greater prospects. — Nikola Tesla

[Pascal Costanza]

Regarding questions from the original post, the Common Lisp community at large seems far from attaining a quorum, let alone consensus.

Common Lisp encourages developing a personal programming style, which may or may not be compatible with styles developed by other Common Lisp hackers.

I’m among those who believe that’s a good thing.

Perhaps this presents an opportunity for maintainers of Quicklisp (perhaps in collaboration with Quickdocs) to introduce a style guide for making their lives easier.

It’s unlikely any such style guide will be adopted by the larger community. There have been such attempts in the past, and I don’t think they succeeded.


Pascal

--
Pascal Costanza

[Daniel Pezely]

Post by Scott McKay
+1
I think we did OK in CLIM: an API package, and an internal package to implement in.
--Scott

Packages are massively overrated.  [...]

Scott,

Are you referring to the earlier CLIM from CMU [1] or McCLIM [2] or
another specific implementation?  My understanding is that "CLIM" today
usually implies the spec, but I've yet to use any of these packages.

What did you or others appreciate most about its package organization?

McCLIM seems to separate API from internals and uses various systems
named `mcclim/foo` and packages named `mcclim-foo` invoked by those
systems.  This is just from glancing at various *.asd files and
package.lisp.

The older CMU version seems to be more fine-grained by separating
platform-specific bits under `impl_dep` subdirectory.

  [1]
http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/gui/clim/0.html
  [2] https://github.com/McCLIM/McCLIM

Thanks,
  -Daniel

[Daniel Pezely]

Post by Ken Tilton
Packages are massively overrated. This is not Java where every frickin
source file is a namespace. There is a certain obsessive
compulsiveness about packages that does nothing but slow developers
down. Well, right, they are a palliative for the OCD disease. But it
*is* a disease, so that does not count.
What part of agile do we not understand? Fences, boxes, categories,
types all invented for their own sake let us bask in our taxonomicity
while getting no code written, and god help the sucker who tries to
use our OCD mess forever battling package issues.
Stop. Wrong way. Go back.

Ken,

How might we use this criticism constructively?

For the intention of having other people use the source code and
possibly being picked up by Quicklisp, what do you propose instead, if
packages are overrated?  My understanding: Quicklisp relies upon ASDF
systems, which in turn builds upon packages by convention. What have I
missed there?


This is separate from being agile.  I see agile methodologies as how to
attain a "release" most effectively while criteria or requirements
continue evolving.

The question of the original post is ultimately about the release as a
discrete artifact.

In my case and for many others, the release means playing nice with
other code fetched by Quicklisp as well as minimizing barriers for other
people to comprehend and reason about the code without involving
original authors or current maintainers.  Using conventional mechanism
such as packages and ASDF systems serves that end.


When you say, "Go back", go back to what?

The days of downloading a tar file-- or worse, a single source file--
from assorted locations in hopes of having it all work together
thankfully ended for the most part.  I've happily traded that era for
repeatable, reliable builds with optional version-pinning and am forever
thankful to Xach for bringing this to contemporary CL users.

Or have I misunderstood your points?

Thanks,
  -Daniel

[Svante v. Erichsen]

Hi!

ASDF does not "build on packages". ASDF loads files, in some of which usually packages happen to be defined, but ASDF doesn't depend on that. You could define an entire system only of symbols in the cl-user (or even keyword) package, and neither ASDF nor Quicklisp would care.

The only concern of packages is organizing symbols. The question is how that should correspond to the file structure (and how that should look like). There is a continuum there, and I guess that it would help to have some resource showing the usual patterns.

Yours aye

Svante

Post by Ken Tilton

Post by Ken Tilton
Packages are massively overrated. This is not Java where every

frickin

Post by Ken Tilton
source file is a namespace. There is a certain obsessive
compulsiveness about packages that does nothing but slow developers
down. Well, right, they are a palliative for the OCD disease. But it
*is* a disease, so that does not count.
What part of agile do we not understand? Fences, boxes, categories,
types all invented for their own sake let us bask in our taxonomicity
while getting no code written, and god help the sucker who tries to
use our OCD mess forever battling package issues.
Stop. Wrong way. Go back.

Ken,
How might we use this criticism constructively?
For the intention of having other people use the source code and
possibly being picked up by Quicklisp, what do you propose instead, if
packages are overrated?  My understanding: Quicklisp relies upon ASDF
systems, which in turn builds upon packages by convention. What have I
missed there?
This is separate from being agile.  I see agile methodologies as how to
attain a "release" most effectively while criteria or requirements
continue evolving.
The question of the original post is ultimately about the release as a
discrete artifact.
In my case and for many others, the release means playing nice with
other code fetched by Quicklisp as well as minimizing barriers for other
people to comprehend and reason about the code without involving
original authors or current maintainers.  Using conventional mechanism
such as packages and ASDF systems serves that end.
When you say, "Go back", go back to what?
The days of downloading a tar file-- or worse, a single source file--
from assorted locations in hopes of having it all work together
thankfully ended for the most part.  I've happily traded that era for
repeatable, reliable builds with optional version-pinning and am forever
thankful to Xach for bringing this to contemporary CL users.
Or have I misunderstood your points?
Thanks,
  -Daniel

[Ken Tilton]

Post by Ken Tilton
Packages are massively overrated. This is not Java where every frickin
source file is a namespace. There is a certain obsessive compulsiveness
about packages that does nothing but slow developers down. Well, right,
they are a palliative for the OCD disease. But it *is* a disease, so that
does not count.
What part of agile do we not understand? Fences, boxes, categories, types
all invented for their own sake let us bask in our taxonomicity while
getting no code written, and god help the sucker who tries to use our OCD
mess forever battling package issues.
Stop. Wrong way. Go back.
Ken,
How might we use this criticism constructively?

Use one package per library. If you want to document your library, document
the public bits.

btw, I did not find the McCLIM idea of a second package for internals
abhorrent.

And the day I let a disaster like ASDF dictate my coding is the day I sign
up for bartender school.

We had an 80kloc CL app that was well divided into packages and it was an
endless source of pain. It took a week but I flattened everything into one
package and we had one bug related to symbol confusion. But then we used
the old 2-3-character prefixing scheme for readabilty and disambiguation.

-kt

[Don Morrison]

Post by Ken Tilton
Use one package per library. If you want to document your library,

document the public bits.

While by and large this does seem to work well, there is often value in
having a second package for the tests, that uses the library’s main
package. Otherwise you can accidentally use internal symbols in your tests
in ways not publicly exposed to real users of your library; on those
occasions when you really do need access to internal symbols in your tests
it seems better to have to explicitly import or qualify them.

[Ken Tilton]

Yep, a test suite is fine, and making it a separate project as well.

-hk

Post by Don Morrison

Post by Ken Tilton
Use one package per library. If you want to document your library,

document the public bits.
While by and large this does seem to work well, there is often value in
having a second package for the tests, that uses the library’s main
package. Otherwise you can accidentally use internal symbols in your tests
in ways not publicly exposed to real users of your library; on those
occasions when you really do need access to internal symbols in your tests
it seems better to have to explicitly import or qualify them.
--
“Each of these views has been held by an important segment of the
cognitive science community, and the two segments do not often
communicate with each other, except sometimes to quarrel.”
– Herbert Simon, /Models of My Life/