On Wed, 27 Aug 2008, Chris Van Dusen wrote:

Agreed.  I saw AppKiDo mentioned in the online documentation, and have been
using it pretty heavily since.
One thing I've been curious about (and may produce on my own, if none
exists) is a cheat sheet for the various ways of interacting with
Objective-C.  Until I get used to it, when to use the appropriate reader
macro is a source of frustration.

All of that said, does anyone have suggestions/recommendations as to how
best to learn to commingle CCL & Objective-C?  I'm fairly comfortable with
Lisp in general.  Would it be worth my while to devote my time to learning
Objective-C, Cocoa, etc. to the point that I'm conversant in those things,
then come back at it from the CCL perspective?

Thanks,
Chris.

When learning CL, there were probably some fairly major concepts
that needed to be absorbed, including:

 - macros
 - syntax (or the near total absence of it)
 - the general notion that everything's a function that returns
   some number of values, and that most primitive operations
   (arithmetic, array/string access) are expressed as function calls
 - closures and higher-order functions
 - special variables and dynamic binding
 - lambda-lists and variadic functions
 - garbage collection (which might or might not be a new concept)
 - CLOS
 - the reader, packages ..
 - the fact that there's a fairly rich library of standard functions
   for dealing with common data structures
 - ...

At some point - when enough of these things make enough sense - one
gets comfortable enough with enough of these concepts to be able to
program productively in CL.  Learning to program in CL probably has
more to do with becoming comfortable with a relatively small set of
core concepts (like those above) than it does with memorizing
reference material describing all of the library functions.  (If you
quizzed me right now, I probably couldn't correctly enumerate all of
the possible standard values that the :IF-EXISTS argument to OPEN can
take; since we all know what keyword arguments are and how they
can be used, we can look up those details in CLHS when we need
to and expect that reference material to make sense.  Well, usually.)

For ObjC and Cocoa, the number of core concepts that we need to
learn in order to start to make sense of things is (mostly) pretty
small; if we want to be able to read ObjC code/examples, there
are some additional syntactic details.  For ObjC, these concepts
include (some of this is a little circular):

 - certain data structures are "classes"; they serve a role
  very similar to that served by classes in CLOS and other
  object systems.  It's possible to add new classes to a
  running ObjC program (just as it is in CLOS), though ObjC
  is less flexible than CLOS/MOP is about redefining/removing
  classes procedurally.  ObjC classes are uniquely named (by
  a string); many of the predefined classes in Cocoa (which
  was derived from NeXT's NeXTStep framework) have names
  that start with the prefix "NS".

  ObjC classes are allocated in foreign memory (often as
  part of a shared library's initialized data.)  A class
  decribes the layout of its instances' instance variables
  ("slots" in CLOS) and describes methods which are specialized
  on its instances.

  Redefining ObjC classes at runtime probably doesn't work
  very well.  In CCL's ObjC bridge, accidentally mousing
  on an unchanged class definition is probably harmless.
  Adding/removing slots won't affect existing instances
  in general, and I don't know/remember what Bad Things
  might happen if the class hierarchy was changed.

 - classes can be instantiated (just as in CLOS and most other
  object systems); this involves allocating a (somewhat opaque)
  block of foreign memory.  If one knows that a block of
  foreign memory contains an ObjC instance, it is possible
  (and cheap) to determine the class of that instance at
  runtime.

 - unlike the case in CLOS - where classes are usually instances
  of the metaclass STANDARD-CLASS - ObjC classes of unique
  metaclasses.  (Among other things, this scheme allows methods
  on classes to be specialized and customized.)  An ObjC class
  is therefore an instance of its metaclass (which is itself
  an instance of a special root metaclass), and it's possible
  to consider any ObjC class, metaclass, or instance as being
  an "ObjC object".  All three kinds of ObjC object are implemented
  as "blocks of foreign memory"; the blocks occupied by classes
  and metaclasses are "permanent" (for the lifetime of the
  application), while those occupied by ObjC instances can be
  freed (and possibly reallocated) when it appears that they
  can no longer be referenced.  Apple is moving toward the
  use of real garbage collection to determine when instances
  can be reliably and automatically freed; traditionally, a
  reference-counting scheme has been used.

  A foreign type corresponding to "pointer to ObjC object"
  is predefined in ObjC; this type is named "id" (or :id from
  CCL).

 - some strings (usually in mixed case and often containing
  embedded and/or trailing colons) name ObjC "messages".  An
  ObjC message isn't quite a first-class object, but it can be
  thought of as being similar to a kind of generic function
  which is specialized on the class of its first argument
  (which must be an ObjC object) and which may contain
  additional arguments.  Those additional arguments may
  be ObjC objects or low-level C pointers or numbers.
  A string which names a message is sometimes called a
  "selector" (in some other ObjC implementations, a "selector"
  is distinct from the string which identifies it, but that's
  a separate issue.)

  The number of colons in a message name typically indicates
  the number of arguments (other than the first, special
  argument) that methods defined on the message should accept.

 - It's possible (and useful) to define methods on any message
  and any ObjC class or metaclass.  A large number of methods
  (on a large number of classes ...) are predefined in Cocoa
  and other ObjC libraries; methdods can also be added at
  runtime.  In order for ObjC code to be able to pass arguments
  (other than the first specialized argument of type "id")
  to and return results from an ObjC method, the (foreign)
  types of those arguments and return value must be specified.
  If a method shadows another method defined in a superclass
  (or is shadowed by a method defined in a subclass), the
  types of those arguments and return value must agree.
  (It's legal for methods defined on disjoint classes to
  have different type signatures; this happens occasionally
  in practice.)

  All methods defined on an ObjC message must take the same
  number of arguments, but (as noted above) the types of
  their non-specialized arguments may differe in some cases.

  It is possible to redefine an ObjC method at runtime; it
  generally works reliably to change the implementation (body)
  of the method, but may not work as well (or at all) to


 - From within the body of an ObjC method M, it's possible to
  invoke whatever method would have been applicable had M
  not been defined.  (Conceptually, this is similar to
  CALL-NEXT-METHOD in CLOS.)

 - In ObjC (as in SmallTalk), invoking a generic function on
  a specialized argument ("the receiver") and some other
  arguments is called "sending a message", and it's written
  in a sort of infix syntax:

[reciever messageName]   if the message accepts no other arguments
[receiver messageNameFirstArg: first-arg-value nextArg: next-arg-value]
otherwise.

  In the first case, the actual message name is "messageName"; in the
  second  case, it's "messageNameFirstArg:nextArg:"

 - Methods can be defined on classes (and therefore applicable to
  their instances) or on metaclasses (and therefore applicable to
  class methods).  When a class method is defined or described, it's
  usually prefixed with a "+" character; an instance method is
  usually prefixed with a "-".

That's a fair amount of stuff that really should be written down
someplace and in greater detail.  A short version is that ObjC offers
an object system with dynamically typed objects and at least some
amount of/some kinds of runtime extensibility, and a sort of
restricted form of (something like) CLOS method dispatch that's based
on the (runtime) class of the first/only specialized argument, and
that ObjC methods typically deal with some combination of run-time
typed Objc objects and statically-typed primitive C things (numbers,
pointers, structures.)

I suspect (I don't know for sure) that the thing that'd help most
people the most in getting up to speed with using Cocoa in CCL is
comfort and familiarity with CCL's FFI.  If that's true (and to the
extent that it is), it's somewhat unfortunate; when he first started
working on it, Randall Beer said that ideally he'd like the bridge to
expose as much of Cocoa as possible while hiding as much of ObjC as
possible.  It's gotten closer to that goal over the years, but I
suspect that a lot of things that people have trouble with could
be blamed on the fact that ObjC (and some of the arcana of dealing
with it) still isn't hidden well enough.  (If this is true, it's
probably the "C" part that's problematic; the "Obj" part seems
to fit a lot better into CL/CCL.)  Looking back at it, I think
that the bridge actually -does- do a good job of hiding a lot
of the problematic C stuff (though admittedly not all of it),
but a lot of it is ultimately nothing more than a few layers
of syntactic sugar around the FFI, and I imagine that it's
still hard to use unless one has some sense of what's going
on underneath.

For learning Cocoa itself ... well, it's a large class library (much
larger than the standard CL library) that provides default behavior
for a lot of things that an application needs to do (implemented as
instances of predefined classes and predefined methods on those
classes), and the general idea is that by subclassing some of those
classes and overriding some of those methods you turn this generic
application into something that offers custom behavior.  It's
generally hard to absorb all of the functionality that's available
in a library by reading the reference manual from cover to cover
(it might help to skim reference material to get a sense of
what's available and to read a few passages that seem interesting.)
It's often helpful - especially when starting out - to have a
specific goal in mind, and then try to learn what you need to
learn in order to achieve that.  After you repeat that process
a few times, it likely starts to get easier, and (as is often
the case) the first foray into something new is the hardest.

Suppose that we had a specific goal of writing an image-viewing
program, like a watered-down version of Apple's Preview.app)
(We could make the example more interesting, but let's not ...)
If we've never done this before in Cocoa, there are clearly
a bunch of specific questions we'd need answered, including:

 - what predefined support is there for loading files containing
  well-known image types (jpg, tiff, gif, png, ...) and displaying
  them ?  If we wanted to construct an image from an unsupported
  file type or procedurally, what's involved in doing that ?

 - how exactly do you display a window and get an image to appear
  in it ?  How do you control what happens when the window is
  resized (e.g., does the image scale ?  do you want it to ?)

 - how do you provide a user interface that allows images to
  be loaded (and other behavior that a more interesting
  application would want) ?

Apple does provide very good overview documentation (in the
form of "guides".)  At the moment, the Cocoa guides are
available at:

<http://developer.apple.com/documentation/Cocoa/index.html>

and looking at the "Graphics and Imaging" section there we find links
to guides describing "Image Views" (sounds promising) and a "View
Programming Guide".  The image view guide is a little terse (it
basically just says that NSImageViews are NSViews that display
NSImages, whatever they are.  The "View Programming Guide" is a lot
more to digest, and if we follow a link to NSImage we'd have even more
to digest.  If we follow the link from the Image View guide to the
NSImageView reference, we see that NSImageViews can be set to allow
cut copy and paste of images as well as drag-and-drop.  That's maybe
not ultimately what we want (if we want to load images from files),
but it may be an easier way to get started.

Let's make a window, put an NSImageView in it, set some options
that enable drag-and-drop and cut-and-paste to work with the
image view, center and display the window, and return the NSImageView
object:

(defun make-image-view-window ()
 "Make a window whose content view is an empty NSImageView.  Allow
the user to drag images to that view, and support transfer of images
to/from the view via the clipboard.  Return the NSImageView."
 (ccl::with-autorelease-pool
  (let* ((rect (ns:make-ns-rect 0 0 300 300))
         (w (make-instance 'ns:ns-window
                           :with-content-rect rect
                           :style-mask (logior #$NSTitledWindowMask
                                               #$NSClosableWindowMask
                                               #$NSMiniaturizableWindowMask
                                               #$NSResizableWindowMask)
                           :backing #$NSBackingStoreBuffered
                           :defer t))
         (image-view (make-instance 'ns:ns-image-view)))
    ;; Set the window's title to a constant NSString (# <at> "...").
    (#/setTitle: w # <at> "Drag or paste an image here!")
    ;; Make the image view respond to drag-and-drop
    (#/setEditable: image-view t)
    ;; and make it support the clipboard
    (#/setAllowsCutCopyPaste: image-view t)
    ;; Make the image view be the window's "content view".
    (#/setContentView: w (#/autorelease image-view))
    ;; Center and display the window.
    (#/center w)
    (#/orderFront: w nil)
    image-view)))

If we call this function, an empty 300x300 window should appear on
the screen and an NSImageView object should be returned.

? (make-image-view-window)
#<NS-IMAGE-VIEW <NSImageView: 0x1279bf50> (#x1279BF50)>
? (defvar *v* *) ; so we can refer to the NS-IMAGE-VIEW later.
*V*

If we then switch to a web browser or other image-viewing application
and copy an image to the clipboard, we can switch back to the IDE and
paste that image into the image view.

I happened to copy a fairly small image:

? (#/image *v*)
#<NS-IMAGE NSImage 0x12795df0 Size={120, 80} Reps=(
   NSBitmapImageRep 0x1279cf30 Size={120, 80} ColorSpace=NSCalibratedRGBColorSpace BPS=8 BPP=32 Pixels=120x80 Alpha=YES Planar=NO Format=2 CGImage=0x1279e340
) (#x12795DF0)>

When I resized the window, the image stayed at 120x80 and happened to
stay centered in the window.  It may be more impressive to let the image
scale itself to fit the view bounds:

? (#/setImageScaling: *v* #$NSScaleToFit)
NIL

Getting images from the clipboard is all well and good, but the original
goal was to display images that were loaded from files.   If only
NSImage had an "initWithContentsOfFile:" method, this would be trivially
easy in most cases.

Oh, wait: it does (as revealed by a little bit of poking around in the
NSImage reference doc.)  ObjC messages whose names start with
"init" are treated specially by the bridge: MAKE-INSTANCE with keyword
args derived from the rest of the message name will (among other
things) send that initialization message to a newly-allocated instance
of the class. So:

(let* ((image (make-instance 'ns:ns-image :with-contents-of-file # <at> "/Library/Desktop Pictures/Nature/Aurora.jpg")))
 (unless (%null-ptr-p image) ; unless the image couldn't be initialized
   (#/setImage: *v* (#/autorelease image))))

We know from an earlier message in this thread (if not otherwise)
how to use NSOpenPanel to select a file, so we're maybe around 75%
of the way towards the goal of developing a simple image-viewing
program.  That's probably enough for an off-topic reply, but it's
worth noting that so far it's been about 20 lines of code and that
NSImage and NSImageView and the rest of Cocoa are doing all of the
heavy lifting.

I'm sorry that this reply is so long and not entirely responsive. To
try to summarize:

 - I think that the best way to learn Cocoa (or anything similar)
  is probably from the bottom up and from the inside out, and that
  it helps to have a concrete, narrowly focused goal and to try
  to learn just what you need to learn in order to achieve that.
  After repeating that process a few times, more stuff tends to
  resonate and make sense.

  It's certainly not practical to go in the other directions,
  as in "here are several hundred classes and a few thousand
  methods, go learn them."

 - From your comments and others, I think that it's fair to say
  that the goal of hiding ObjC (the C part of it, especially)
  hasn't been fully reached.   It's still sometimes necessary
  to allocate foreign objects and pass pointers to them by
  reference, and that sort of thing is disconsonant with what's
  otherwise a very high-level and powerful framework (Cocoa.)
  That stuff is also inherently hard and un-lispy.

One of the problems that I personally have here is that while
I can believe that there are aspects of CCL's FFI that are hard
to use and/or understand, it's hard for me to guess exactly
what those things are (since it all makes perfect sense to me
for some reason.)