Tangled in the Threads

Jon Udell, June 13, 2001

Nature vs Nurture

The deep magic of Zope acquisition

Understanding Zope's acquisition mechanism is a challenge. Fortunately, you can reap the benefits without knowing much about how it works.

A newsgroup thread which has influenced two earlier columns -- on document engineering and the future of the file system -- continues to unfold in thought-provoking directions. Of course, as Rich Kilmer reminds us, these paths were trodden long ago:

Rich Kilmer:

I'm reminded of the Memex idea of Vannevar Bush (see http://www.theatlantic.com/unbound/flashbks/computer/bushf.htm). All the way back in 1945 he realized that the ultimate system was one which allowed the storage of associations between objects.

To me, context is the collection of meaningful associations between objects. An object's ontology (what it is) defines its basic attributes and behavior, but it exists in a web of contextual links. If both of these things were standardized, it would allow tools that operated on specific types of ontological objects (messages, or people, or orders) to interoperate via the contextual linkages. Order management systems would then just focus on order management, and messaging systems would focus on messaging...but both would extend a common context. Then new tools could emerge that allowed us to benefit from the context between and around these things.

Although the ideas in that prescient paper by Vannevar Bush have been kicking around for a long time, the technologies that enable us to implement them may only now be emerging. One of these, as Patrick Phalen points out, is acquisition, at once the most powerful and mystical feature of Zope:

Patrick Phalen:

Jon has elsewhere brought up Zope's concept of containment through "Folderishness." Zope folders are also informed by notions of Environmental Acquisition and its powerful means of context control, perhaps best explicated here:

http://www.cs.technion.ac.il/Labs/Lpcr/publications/lpcr9507.html

Here is the abstract of that seminal paper, Environmental Acquisition - a New Inheritance Mechanism, by Joseph Gil and David Lorenz:

Nature vs. Nurture? The debate has obsessed the minds of psychologists and philosophers for many years. However, for the object-orienteer, it has never been a problem: an object inherits all of its properties. In this work we ask if an object should not be subject to environmental effects. We answer this question in the affirmative by demonstrating many cases in which the character of an object must be affected by the environment it is put in. We present a new abstraction mechanism - Environmental Acquisition - which allows a component to inherit properties from its enclosing composite(s).

Why would this technique be useful? The authors point out that "there are abundant cases of containment hierarchies where contained objects have different behaviours depending on their surrounding environment." Although acquisition as an object-oriented programming discipline is wildly complex and esoteric, its uses are common and familiar. Here are two cases they cite. In a GUI system, the coordinates of a button within a dialog box are acquired by virtue of the button's containment within the dialog box, not by virtue of an inheritance relationship. In LATEX, emphasized text normally prints as italic, but prints as roman if contained within an emphasized block.

I now must confess that although I am a Zope user, and have written a lot about Zope, Patrick's reference was my first introduction to the formal concepts underlying acquisition in Zope. That research, Patrick says, suggested to Zope architect Jim Fulton a way "to handle authorization issues that inheritance couldn't."

Patrick Phalen:

Zope sets out to implement a fine-grained, hierarchical security model. Zope is an object-oriented system, so the primary operation is attribute access. Zope allows access to any attribute with a non-elementary datatype to be protected individually by a permission. Zope does not associate permissions with individual users, but rather with roles -- an abstraction. If security were built out of "is-a" inheritance stuff, users could conceivably override attributes of security-defined objects from higher up. Acquisition works the other way around. You can't have an object at the top whose contents can be overridden as you go down. Acquisition permits aggregating permission data in a global and secure way.

This explanation, Patrick hastened to add, only scratches the surface. For more information, he referred us to the environmental acquisition page maintained by David Lorenz, which gathers a number of useful links, and includes these pithy comments:

David Ascher:

It's very cool, somewhat non-trivial to use right (it affects the way one designs massively), and requires a fairly deep brain-tweak. Still, worth knowing about when designing a new framework where containment is such an important notion. It's one of the few "new" concepts in programming that I've seen in the last few years which I think has huge potential for simplifying complex designs.

Samual A. Falvo II:

What Zope describes in several pages of text I can describe in two sentences: "If the file or variable isn't defined in the current folder, check the parent folder. If it's not there, continue checking parent folders until you can't anymore." This is also true of properties as well as files.

These two comments nicely bracket my experience with acquisition in Zope. On the one hand, it does require a "deep brain-tweak" which I feel I have yet to really achieve. On the other hand, I've been using acquisition without really thinking much about it. In this respect, Zope reminds me of Perl. Both are large and complex programming environments. But when I started with Perl, I was able to get useful results without really understanding more than a fraction of all that Perl was and could do. And it's been the same way with Zope. It has the excellent property of "discoverability" -- that is, you can build simple things in simple ways, without knowing about the deep magic, and then gradually over time begin to discover and apply the deep magic.

Examples of acquisition

Some simple examples of acquisition can been seen in an application I wrote to help a team of editors collaborate on the production of a magazine. The application uses ZODB (Zope's object database) to store a tree of contents that looks like this:

2001-08
    article_1
        draft_1
        draft_2
        image_A
        image_B
    article_2
2001-07
    article_1
    article_2

The bolded items are Folders. There's a Folder for each month's issue, and within that, a Folder for each article in the issue, containing one or more Files or Images. The user interface of the application is based on Zope's tree tag. Here's a simplified view of the DTML logic that expresses the containment hierarchy in HTML:

<dtml-tree branches_expr="objectValues(['Folder', 'File','Image'])" reverse sort=id>

  <dtml-if "meta_type=='Folder'"> 

   <dtml-if "isIssueFolder(_['title_or_id'])==0"> 
     [<a href="<dtml-var tree-item-url>/addFile">add file</a>]
     [<a href="<dtml-var tree-item-url>/addImage">add image</a>]
   </dtml-if>

  </dtml-if> 

</dtml-tree>

The isIssueFolder() method is called once for each Folder in the containment hierarchy, and passed an argument that picks out the title_or_id property of the current object. The convention here is that when that property doesn't match the regular expression 20\d\d\-\d\d (which is the test implemented by isIssueFolder), the Folder represents an article. In these cases, the DTML logic decorates the emitted HTML node with links used to add files, or images, to that folder, like so:

2001-08
    article_1 [add file] [add image]
        draft_1
        draft_2
        image_A
        image_B
    article_2 [add file] [add image]
2001-07
    article_1 [add file] [add image]
    article_2 [add file] [add image]

The addresses of these links are formed by means of the expression <dtml-var tree-item-url>, which yields URLs like these:

2001-08/article_1/addFile

2001-07/article_2/addImage

addFile and addImage are DTML Documents that present HTML forms used to upload files or images. Here, for example, is the preamble to addFile:

<p>Add a file to 
<dtml-var "PARENTS[1].title_or_id()">/<dtml-var "PARENTS[0].title_or_id()">
<FORM ACTION="addFileToArticleFolder" METHOD="POST" ENCTYPE="multipart/form-data">

When you call the URL 2001-08/article_1/addFile, PARENTS[0] is the Folder article_1 and PARENTS[1] is the Folder 2001-08. But this Folder hierarchy does not, in fact, contain the DTML Document addFile. Let's expand the scope to show more of the Folder hierarchy:

Production
    addFile
    addImage
    2001-08
        article_1
            draft_1
            draft_2
            image_A
            image_B
        article_2
    2001-07
        article_1
        article_2

So while addFile is contained within Production, it is acquired by all of Production's subfolders. When called from such contexts, the PARENTS are acquired in a context-sensitive way.

The External Method addFileToArticleFolder, which is wired to the ACTION attribute of the form, exhibits similar context sensitivity. Here's the essence of that method:

def addFileToArticleFolder(self,REQUEST):
    self.manage_addFile(REQUEST.file.filename,REQUEST.file.read(),'')
    return REQUEST.RESPONSE.redirect(REQUEST['BASE2'])

When this method is called from a context like 2001-08/article_1/addFile, the self object is the Folder article_1, and so the file is added to that Folder.

A glimpse of the deep magic

All this I absorbed osmotically, from other examples of Zope usage, without really having much of a clue about the mechanics of acquisition, or its algebra as explained in Jim Fulton's now-famous lecture at IPC8, and also in another write-up by Dieter Maurer.

Until recently, despite my ignorance of these matters, I never ran into the kinds of conflicts that can occur when you mix the containment flavor of acquisition with the context flavor. But last week, while adding a new feature to my magazine-production application, I finally dipped my toe into the waters of the deep magic.

Here's the background. When the magazine is published to the web, I base each article's URL on a short label. Sometimes, an article has to go to print with a "forward reference" to a future article whose URL is not yet assigned. So, I decorated the <dtml-tree> with a new per-article-folder link used to pre-assign that label in these cases. The link invokes code that sets an url_label property on the article's Folder. I also added to the <dmtl-tree> logic some code to display the value of that property.

To test my ability to display the property, I used a DTML Document, called test, containing just this expression:

<dtml-var url_label>

But, no matter from which context I called test, the result was always 'xxx'. Here's why. At some point, I had set that property on the root Folder, Production, and given it the value 'xxx'. Subsequently, my per-article-Folder code was (as it should) adding other url_label properties. The resulting structure was:

Production          <- url_label = xxx
    addFile
    test
    2001-08
    2001-08
        article_1   <- url_label = abc

Why would the URL 2001-08/article_1/test print the value 'xxx'? Because test acquires the property in two ways: by containment, and by context. And, because -- as Jim Fulton's acquisition algebra paper explains -- the former takes precedence over the latter.

The fix, in this case, was simply to delete the unneeded url_label on the Production Folder. But suppose it had to exist, for some other reason. How could you choose context over containment? As it turns out, my standalone DTML Document test was a red herring. Within the <dtml-tree> that's the engine of this application, the contextual property is the one that's found first and used.

When you write a Python Script -- the newer, more integrated, but less powerful and dangerous version of an External Method -- the choice becomes explicit. Such a script receives two namespaces, appropriately called context and container. A Python Script called testPyScript which contains the following code:

if (which=='context'):
  return context.url_label
else:
  return container.url_label

produces these results:

2001-08/article_1/testPyScript?which=context     -> abc
2001-08/article_1/testPyScript?which=container   -> xxx

Whew! This is indeed a bit of a "brain-tweak." Quite honestly, I still feel like a novice when it comes to understanding this stuff. But as a novice, I'm nevertheless able to do some really useful things with Zope. Powerful magic like acquisition is one reason why. Brilliant integration of that magic into a web application server is the other.

Final note

I asked Amos Latteier and Michel Pelletier, authors of The Zope Book (available online and, soon, in print), to review this column for accuracy. Michel offers this useful clarification:

The "wrapping" that acquisition does is actually a cleaner, simpler model that just happens to end up turning into a complex pile of algebra if you care. Most of the time, it works just like people think it does.

One easy way to think about it is, you can acquire things from objects if you are in their context. You're in their context if they are an element in your URL path.

---

Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He is the author of Practical Internet Groupware, from O'Reilly and Associates. Jon now works as an independent Web/Internet consultant. His recent BYTE.com columns are archived at http://www.byte.com/index/threads

This work is licensed under a Creative Commons License.