geo/tech/things

Y Combinator

erilem — Thu, 02 Jul 2009 19:08:46 +0000

I’ve been reading “The Little Schemer” from Daniel P. Friedman and Matthias Felleisen. Very interesting reading.

The nineth chapter introduces the Y Combinator function, a pretty interesting beast! Quoting Crockford : “one of the most strange and wonderful artifacts of Computer Science”. As a primer, here’s how the Y Combinator looks like (in Python):

def Y(func):
    return (lambda f : f(f))
     (lambda f : func(
          lambda x : (f(f))(x)))

Looks scary, doesn’t it? (It did scare me when I first saw it at least.)

The Y Combinator creates a recursive function from a non-recursive function that looks like the recursive function one wants to create. The Y Combinator can for example be used to obtain recursive functions from anonymous functions, which, with most programming languages, cannot be recursive.

This blog post proposes defining the Y Combinator function in Python.

Goal: find the function Y (the Y Combinator) such that

fact = Y(like_fact)

where:

fact is the factorial function
like_fact is defined as follows:

def like_fact(r):
    def f(n):
        if n < 2:
            return 1
        else:
            return n * r(n - 1)
    return f

So Y takes a non-recursive function (which can theoritically be expressed as an anonymous function) that looks like the recursive factorial function and returns the factorial function.

You may have noticed thay our like_fact function is not expresed as an anonymous function. This is because Python does not allow us to do it: the inner function f cannot be defined with lambda because it includes conditional statements, the outer function like_fact cannot be defined with lambda because it includes an inner function that isn’t defined with lambda. Using JavaScript the like_fact function would be:

function(r) {
    return function(n) {
        return n < 2 ? 1 : n * r(n - 1);
    };
}

We start our demonstration from the following statement:

fact = notlike_fact(notlike_fact)

where notlike_fact is:

def notlike_fact(r):
    def f(n):
        if n < 2:
            return 1
        else:
            return n * (r(r))(n - 1)
    return f

Now we rewrite the above statement using lambda:

fact = (lambda f : f(f))
          (notlike_fact)

Now we can extract like_fact and rewrite the statement as (maybe the most difficult step):

(lambda f : f(f))
 (lambda f : like_fact(
      lambda x : (f(f)(x)))

We can now write the Y function:

def Y(func):
    return (lambda f : f(f))
     (lambda f : func(
          lambda x : (f(f))(x)))

And we have:

fact = Y(like_fact)
assert fact(1) == 1
assert fact(2) == 2
assert fact(3) == 6
assert fact(4) == 24
assert fact(5) == 120

Cool, no?

Obviously Y applies to other recursive functions, as an example let’s apply it to Fibonacci:

def like_fibo(r):
    def f(n):
        if n <= 2:
            return 1
        else:
            return r(n - 1) + r(n - 2)
    return f

fibo = Y(like_fibo)
assert fibo(1) == 1
assert fibo(2) == 1
assert fibo(3) == 2
assert fibo(4) == 3
assert fibo(5) == 5
assert fibo(6) == 8

Testing

erilem — Tue, 30 Jun 2009 11:37:10 +0000

I’ve been reading about testing. Here are a few words on my thoughts about testing.

From my reading and understanding there are three types of tests:

Unit tests: a unit test tests a single function (e.g. an object method). A unit test must take care of isolating the tested function from the functions the tested function normally relies on (when executed outside any test).
Integration tests: an integration test tests if two or more dependent functions correctly work together.
User-acceptance tests: a user-acceptance tests whether a given function provides the behavior its users expect. User Interface tests belong to this type.

These three types of tests are complementary, they all have their importance when testing an application.

In OpenLayers, GeoExt, and MapFish (its JavaScript library), we provide unit and integration tests, and actually don’t distinguish whether they’re of the unit or integration type (they’re all referred to as unit tests, which is fine I think). Not providing user-acceptance tests makes sense, as OpenLayers, GeoExt and MapFish are libraries as opposed to applications. The three libraries come with examples that in some way are user-acceptance tests. (In OpenLayers we’ve attempted to create actual user-acceptance tests, but developpers haven’t paid much attention to them, possibly their scopes and goals haven’t been well defined.)

Applications built with OpenLayers and/or GeoExt and/or MapFish instantiate classes from these libraries. Often, most of their code doesn’t include actual logic, and from that regard writing unit and integration tests for such applications doesn’t make sense. However, as User Interfaces, these applications would deserve user-acceptance tests.

Providing automated User Interface tests is in my opinion a very difficult task, and I’d be very interested in having feedback from others on that.

MapFish and GeoExt

erilem — Sun, 19 Apr 2009 20:40:11 +0000

Matt Priour recently asked about the future of the client part of MapFish, and more specifically whether it will be replaced by GeoExt. This is actually a question that every MapFish user should be asking . Anyway I thought an answer to that question could make a post on my blog. There it is.

The short story: the client part of MapFish will not be replaced by GeoExt.

Now the longer story. As of today the client part of MapFish includes OpenLayers, Ext, and the MapFish JavaScript lib. The latter is itself composed of two parts: core and widgets.

core includes classes that are independent of Ext; most of them extend OpenLayers classes like OpenLayers.Control, OpenLayers.Protocol, OpenLayers.Strategy, etc. For example the client-side implementation of the MapFish Protocol is part of core.
widgets includes Ext-based classes, mostly GUI components (but not only, the FeatureReader and stuff are part of widgets). widgets also has stuff that’s directly related to the server side of MapFish, the print widgets are a good example.

GeoExt will not replace core, nor will it replace the widgets components that rely on MapFish web services. But basically every new Ext-based component that isn’t tied to any server-side stuff is going into GeoExt.

In addition to OpenLayers and Ext, MapFish will include GeoExt. We had initially planned to integrate GeoExt into MapFish earlier, but finally decided to let things settle down a bit in GeoExt before doing the integration. We’re currently doing that integration, and we will gradually be deprecating classes as their equivalents are added into GeoExt. For example, the work on FeatureRecord, FeatureReader and FeatureStore we’ve been doing in GeoExt will deprecate the FeatureReader, FeatureStore and LayerStoreMediator classes in the MapFish JavaScript lib.

Also, MapFish, as a framework, aims to provide an integrated solution. For client-side development, this means that the developer doesn’t need to download Ext, OpenLayers and GeoExt, install them within his application, and think about the organization of his application. Instead, we want that applications created with the MapFish framework are well organized from their creations; with the Ext, OpenLayers, GeoExt and MapFish libs ready, with the JavaScript build tool ready, with the unit test suite ready, etc. I guess I will cover this topic in a later post…

Wooo, two posts in two days, scarry…

Additions to the MapFish Protocol

erilem — Sat, 18 Apr 2009 21:55:48 +0000

We recently added new stuff to the MapFish Protocol.

As a refresher, let’s first take a look at what the MapFish Protocol had before the new additions.

(Note that you’d need the JSONovich FireFox extension to see the output of the examples given below in your web browser.)

Geographic query params

box={x1},{y1},{x2},{y2}: the features within the specified bounding box
geometry={geojson_string}: the features within the specified geometry
lon={lon}&lat={lat}&tolerance={tol}: the features within the specified tolerance of the specified lon/lat

Examples:

Limiting and Sorting

limit={num}: the maximum number of features returned
offset={num}: the number of features to skip
order_by={field_name}: the name of the field to use to order the features
dir=ASC|DESC: the ordering direction

Examples:

The new params

no_geom=true|false: so that the returned feature has no geometry (“geometry”: null)
attrs={field1}[,{field2},...]: to restrict the list of properties returned in the features
queryable={field1}[,{field2},...]: the names of the feature fields that can be queried
{field}__{query_op}={value}: filter expression, field must be in the list of fields specified by queryable, query_op is one of “eq”, “ne”, “lt, “le”, “gt”, “ge”, “like”, “ilike”

And now an example combining all the new parameters:

http://demo.mapfish.org/mapfishsample/trunk/summits?queryable=name,elevation&name__ilike=col&elevation__gte=3500&attrs=name,elevation&no_geom=true

The above query returns a GeoJSON representation of the summits whose names include “col” and whose elevations are greater than or equal to 3500. The returned features have no geometry and their attributes include “name” and “elevation” only.

Not including the geometry in the features makes the parsing in the browser much faster, so for cases where the geometries aren’t needed this is a big win.

Credits for the “queryable={field}&{field}__{query_op}={value}” syntax goes to FeatureServer!

Secure TileCache With Pylons and Repoze

erilem — Sun, 15 Feb 2009 18:14:18 +0000

This post shows how one can secure TileCache with Pylons and Repoze.

In a Pylons application one can run a WSGI application from within a controller action. Here is a simple example:

    class MainController(BaseController)
        def action(self, environ, start_response):
            return wsgiApp(environ, start_response)

TileCache is commonly run from within mod_python. TileCache can also be run as a WSGI application, therefore it can be run from within the controller action of a Pylons application. Here’s how:

    from TileCache.Service import wsgiApp

    class MainController(BaseController)
        def tilecache(self, environ, start_response):
            return wsgiApp(environ, start_response)

Pretty cool… But it gets really interesting when repoze.what is added to the picture. For those who don’t know repoze.what, repoze.what is an authorization framework for WSGI applications. repoze.what now provides a Pylons plugin, making it easy to protect controllers and controller actions in a Pylons application. Here’s how our tilecache action can be protected:

    from TileCache.Service import wsgiApp
    from repoze.what.predicates import has_permission
    from repoze.what.plugins.pylonshq import ActionProtector

    class MainController(BaseController)
        @ActionProtector(has_permission('tilecache'))
        def tilecache(self, environ, start_response):
            return wsgiApp(environ, start_response)

With the above, anyone who tries to access /tilecache will have to be granted the tilecache permission. Otherwise, authorization will be denied.

TileCache is secured!

People often want finer-grained authorization, like give certain users access to certain layers. With Pylons’ routing system this can be easily and elegantly achieved using repoze.what, I will show that in a later post.

FeatureServer Versus MapFish Server

erilem — Fri, 31 Oct 2008 15:10:56 +0000

I thought I could say a few words on the differences between FeatureServer and MapFish Server.

First, FeatureServer and MapFish Server have similarities. They share a similar, REST-based protocol, for creating, reading, updating and deleting features.

The main difference between the two: FeatureServer is a standalone application, MapFish Server is a web-mapping development framework.

FeatureServer is perfect for very rapidly setting up editable layers, with no custom needs. MapFish Server is good if you work on a customer project, with specific, customer-oriented needs; MapFish
Server provides a complete development framework, which, thanks to the great components it relies on (Pylons, SQLAlchemy, Shapely, etc.) allows to write high-quality and maintainable code.

Two different goals.

MapFish 1.0

erilem — Fri, 17 Oct 2008 07:58:39 +0000

MapFish 1.0 is out!

The things I really like in MapFish 1.0:

The PDF Printing Library. Everyone wants to print maps, Patrick has turned everyone’s dream into reality. The PDF Printing Library is great, it supports fancy stuff like vector rendering, map rotation, legend, etc. See this page to know more. And there’s more to come, support for printing map annotations is about to make it into trunk.
MapFish Server implementation of the MapFish Protocol for creating, reading, updating and deleting features. We still need to add a feature editing widget to MapFish Client, the feature editing panel, currently demo’ed here, will probably make it into trunk soon.
MapFish Client relying on the OpenLayers protocol abstraction. With that every MapFish Searcher component can work with the MapFish Protocol as well as with any other protocols supported by OpenLayers (e.g. Gears, WFS).
The feature reader, and mediator components we have added to widgets/data. These are core classes to bridge OpenLayers and Ext. And by the way, these classes will probably represent the first bits put in GeoExt; more on that later…
The API doc that covers both MapFish Client and OpenLayers.

There are other stuff in MapFish 1.0, the above are just the ones I care the most about.

This blog

erilem — Wed, 14 Nov 2007 21:23:03 +0000

My first post to tell what I’m up to with this blog. I’ll be talking about technical stuff mostly related to my activities as a developer at Camptocamp.

Expect me to talk about MapFish – currently my favorite project – and its best friends OpenLayers, the gispython project, and Pylons. Without these great projects, MapFish couldn’t exist.

I’ll also be talking about the Spatial Data Integrator, an opensource spatial ETL.

And hopefully lots of other interesting technical stuff.