Jon Udell: Paying for Web Services

Tangled in the Threads
Jon Udell, August 22, 2001
Paying for Web Services

Benefits include language-neutrality and convenience

Web services and generic local components are two sides of the same coin. Both can be powered by free software, and both can create business opportunities.

When technology hype reaches a crescendo, I instinctively reach for a touchstone -- a humble example that grounds me in reality. This week, as I was almost deafened by the uproar surrounding web services, I found one such touchstone. Here's the scenario. I maintain a Zope-based extranet site for a small group of collaborative knowledge workers. Part of the site uses the Zope calendar tag to keep track of events, and I was asked to arrange for various folks to be notified, by email, on the day prior to these events.

There is, I'm aware, a Zope scheduler, but so far I've been to lazy to try it. I'm running Zope on Linux, and -- thanks to Zope's excellent URL-line interface -- simple Perl scripts kicked off from my cron file work fine for a variety of scheduled actions. The problem reduced to:

Fetch tomorrow's date, in yyyy/mm/dd format.

Fetch the calendar entry for that date. (It's a Zope property keyed to the yyyy/mm/dd string).

If there was an entry for that date, fetch it and send it to a list of email addresses.

Perl's built-in date/time functions are somewhat limited, but there are lots of CPAN date/time modules, including the formidable Date::Manip. In this case, not atypically, the extranet site is hosted by an ISP that omits most of CPAN -- including that module -- from its configuration of Perl. As I mentioned in my newsgroup recently, Perl 6 happily wants to tackle this problem by essentially requiring more complete distributions of Perl.

The reflexive thing for a Perl programmer to do in this situation is to acquire the module and install it locally. But that's more effort than this tiny problem seemed to justify. If I were going to install anything locally, why not something like SOAP::Lite, which would then (in theory) give me ad-hoc access to all sorts of functionality I might need, by way of SOAP and/or XML-RPC?

So, out of curiosity, I checked sites like http://www.xmethods.org/ for a callable web service that would turn 2001/08/13 into 2001/08/14, or 2004/02/28 into 2004/02/29. Nothing readily emerged. Somebody recently joked that the ratio of SOAP/XML-RPC implementations to SOAP/XML-RPC callable services is scarily close to one. In fairness, I didn't conduct an exhaustive search, and I expect (and hope) this column will provoke mail that will point out services I didn't find. In any case, here were the steps to the path-of-least-resistance solution that I arrived at. The process is, I suspect, rather typical, and reveals much about how we find and apply programming resources.
I recalled having used, or read about, a Python DateTime module, and wondered if came standard with the Python distribution I was using. But it didn't.

I searched Google for Python and DateTime. All the references pointed back to Zope. Aha! It was part of Zope's Python distribution, evidently. This was good news.

I considered writing my cron script in Python rather than Perl. But, being no Python expert and vague on the procedure for installing modules, I lazily chose to leverage the fact that the module was already in the Zope kit I was using.
I wrote three lines of code:
d=DateTime()
tomorrow=d+1
return tomorrow.Date()
I put those three lines of code into a Zope Python script, which I named tomorrow. Now Zope's URL-line API was extended to support the URL http://www.xxx.yyy/tomorrow, which returns strings like '2001/08/22'.

Finally, I wrote the notifier in Perl as planned. It calls Zope's URL-line API twice, once for tomorrow's date, and again for that date's calendar entry.
This is, I'm the first to admit, rather ridiculous. But I'll bet that many of you have trodden similar paths, and will again. Let's examine some of the assumptions behind the scenario I've just outlined.

I'm a Perl programmer. No I'm not. Well, OK, I am, but only to the extent that there are some things I need to accomplish for which Perl offers the path of least resistance. I am also not a Python programmer in exactly the same way.

Perl's greatest asset is CPAN. True, but with qualifications. As we've seen, the non-availability of CPAN modules on ISP-hosted sites can be an obstacle. So can the non-availability of CPAN modules for Windows, a situation that has improved greatly but is still not fully resolved.

You can do everything with Zope and Python. True. You can also do everything with any language/app-server combo. I find Zope and Python to be a terrific combo, but the icing on the cake has always been Zope's URL-line (and yes, XML-RPC) integration hooks.

There's a module for every need. Well, there are a lot of great modules floating around. But for a variety of reasons, they're not as easy to find and apply as they might be. In Perl space, deployment issues notwithstanding, you can burn up a lot of time trying various CPAN modules before you find one that does what you need. What Perl programmers know, arguably, has as much to do with the availability and true applicability of Perl modules as with the syntax of the language. Crossing languages, it's even harder. As a relative newbie in Python space, I am much less aware of its module support. In the above example, the module I needed was already included in Zope, and needed only three lines of code to be applied for my purpose. All that stood in the way was awareness: of the module, of its capabilities, of its existence within Zope.

XML-RPC/SOAP web services are the one true way. Well, they are one true way, but there is never just one. That applies to programming languages, and it applies to component architectures. As I noted last week, there is often more value in getting XML out of services, than in sending it in. Sometimes, there is no real value in getting XML out, either. In my humble example, or in the ever-popular stock quote example which has become the 'Hello, world' of web services, it's not clear why you need to wrap layers of infrastructure around a single return string.

A business model for web services

As a thought experiment, imagine a subscription site that offers a whole bunch of CPAN-module functionality as a set of web services -- callable from XML-RPC, SOAP, and where appropriate, URL-line. One of the obvious benefits would be the language-neutrality of these services.

When this point came up in newsgroup, in the context of discussion about SOAP and the .NET Common Language Infrastructure, there was the following objection:

Randall Parker:

I don't want to program in n languages and mix them together. I've rarely seen projects where we wanted to blend code from a lot of different languages. When someone tells me that CLI will let me blend Eiffel and C++ and Python I have to ask why would I want to do that?

And here was the chorus of responses:

Alan Shutko:

Let's say you want to make an app with a KDE GUI using a Java DB library, all written in Python for ease of development. With the CLI, you don't need to have wrappers for every library in every language you might want to use.

Geoff:

Components! If I write a .NET component, you should be able to use it. It shouldn't matter in the slightest what language I used to create the component, it should just work.

Greg Wilson:

I'd estimate that 50-75% of the effort in the product I'm working on these days is integration (a.k.a. "duct tape and prayer"). I can point at lots of places where the widget I need exists in Java, but I'm writing in C++ or Perl, or vice versa. If the CLI simplifies interop, a lot of big shops are going to be very, very happy with it...

Jeffrey P Shell:

Imagine writing a log analysis tool in Perl that responds to SOAP requests and returns various statistics. Imagine being able to call that from Mathematica or MATLAB and being able to generate truly effective graphs and doing wild statistical analysis that such a tool excels at. Imagine being able to turn a powerful client tool such as GoLive or Dreamweaver into an effective BLOGGER editor by communicating directly with a blogger by SOAP or even XML-RPC to be able to pull up recent edits, set different flags on published stories.

There were, to be fair, two parallel threads going on here, one about the .NET CLI for client-side code, and one about SOAP for distributed services. With respect to the latter, Randall's rebuttal was, quite rightly, "Who's going to pay for it?" In other words, who's going to make callable web services freely available on the Net?

This is, of course, precisely why services such as AltaVista and Amazon, which are quite widely screen-scraped by robotic tools, do not offer formal web-services APIs. No advertiser-supported eyeballs see SOAP calls.

From a cost of ownership perspective, though, I think I can make a great case for leasing access to a collection of services that, while in fact free, would cost me a whole lot of effort to find/install/configure/maintain.

It's already true, today, that I recommend and use commercial services (indexing/searching, list management) which I could alternatively implement from scratch using free software that I know how to acquire and use. Why? My time is better spent building things on top of this infrastructure, rather than recreating it.

For subscribers to a hypothetical collection of callable web services, language neutrality is partly just a time saver. When you call a Perl-based XML-RPC or SOAP service from Python, the results show up in a native Python data structure, no translation required. But the model makes sense even when you're not crossing language boundaries. How much time is spent just finding the module that you need, then installing and configuring? I can easily imagine consuming Perl-based services from Perl solely for the convience of skipping the hassle of module acquisition. Could I justify a $10/month subscription to a robust collection of such services? I think so.

This leads us back to the distinction between client-side code and distributed services. Even when built using free software, web services can't be free because, as Randall points out, somebody has to pay to host them. I regard this as a golden opportunity for open source software. One of the answers to the nagging question "How, exactly, do you make a business out of free software?" should be "By leasing services." ISPs of course do just this; the model can and should be broadened. But the leasing model has at least three disincentives.

Because time spent acquiring/configuring software is often not accounted for, it can be hard to justify software leasing on the basis of avoided cost.

A leasing arrangement probably doesn't spell out a clear exit strategy.

Distributed services introduce more points of failure than centralized services.

For all three reasons, there's still a strong incentive to acquire services in the form of installed modules, and use them locally. But does this mean that there can be no business opportunity for open source software in this realm? I don't believe that. ActiveState has already explored this territory with, for example, its Visual Package Manager, which helps manage the collection of Perl modules installed on a system. A generic component system, such as the CLI envisions, should greatly expand such opportunities.

Distributed web services and local CLI-style components are really two sides of the same coin. Both should build on free software foundations. Both should also afford business opportunities.

Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He is the author of Practical Internet Groupware, from O'Reilly and Associates. Jon now works as an independent Web/Internet consultant. His recent BYTE.com columns are archived at http://www.byte.com/tangled/

This work is licensed under a Creative Commons License.