Reading paged LDAP results with PHP is a show-stopper

I was writing the schema introspection code for Zend_Ldap when I came around a problem with Active Directory’s MaxPageSize restriction. By default Active Directory allows only 1000 items to be returned on a single query, a number which is easily exceeded when reading an Active Directory’s classes and especially attributes from the schema tree. OK – one option would be to increase the MaxPageSize variable, but as the component should be usable on every Active Directory server I couldn’t go for that.

The second option that seemed possible makes use of the paged result sets that Active Directory returns on a query. This way led me into the world of LDAP server controls and deep into the ext/ldap source code. There is astonishingly little information on the topic of paged result sets and LDAP server controls in respect of PHP and ext/ldap. To be honest I assume that only one person really looked into this area seriously and even came up with a solution: Iñaki Arenaza (Blog, Twitter, Facebook, LinkedIn). His information provided here is the foundation of this article – the discoveries are absolutely not my work, they are all based on what Iñaki Arenaza dug out. I just wanted to bring a little light into this very specific topic (and summarize what I’ve answered on stackoverflow.com).

To make it short right from the beginning: it’s currently not possible to use paged results from an Active Directory with an unpatched PHP (ext/ldap).

Let’s take a closer look at what’s happening.

Active Directory uses a server control to accomplish server-side result paging. This control is described in RFC 2696 “LDAP Control Extension for Simple Paged Results Manipulation” . LDAP controls, which come in the flavors “server” and “client”, are extensions to the LDAP protocol to provide enhancements – result paging is one example, password policy is another one. Generally ext/ldap offers an access to LDAP control extensions via its ldap_set_option() and theLDAP_OPT_SERVER_CONTROLS and LDAP_OPT_CLIENT_CONTROLS option respectively. To setup the paged control we do need the control-oid, which is 1.2.840.113556.1.4.319, and we need to know how to encode the control-value (this is described in the RFC). The value is an octet string wrapping the BER-encoded version of the following SEQUENCE (copied from the RFC):

realSearchControlValue ::= SEQUENCE {
    size    INTEGER (0..maxInt),
                     -- requested page size from client
                     -- result set size estimate from server
    cookie  OCTET STRING
}

So we can setup the control prior to executing the LDAP desired query:

$pageSize    = 100;
$pageControl = array(
    // the control-oid
    'oid'        => '1.2.840.113556.1.4.319',
    // the operation should fail if the server is not able to support this control
    'iscritical' => true,
    // the required BER-encoded control-value
    'value'      => sprintf ("%c%c%c%c%c%c%c", 48, 5, 2, 1, $pageSize, 4, 0)
);

This allows us to send a paged query to the LDAP/AD server. But how do we know if there are more pages to follow and how do we have to send a query to get the next page of our result set?

The server responds with a result set that includes the required paging information – but PHP lacks a method to retrieve exactly this information from the result set. In fact ext/ldap provides the required function (ldap_parse_result()) but it fails to expose the required seventh and last argument serverctrlsp from the C function ldap_parse_result() in the LDAP API, which contains exactly the information we need to requery for consecutive pages. If we had this argument available in our PHP code, using paged controls would be straight forward:

$l = ldap_connect('somehost.mydomain.com');
$pageSize    = 100;
$pageControl = array(
    'oid'        => '1.2.840.113556.1.4.319',
    'iscritical' => true,
    'value'      => sprintf ("%c%c%c%c%c%c%c", 48, 5, 2, 1, $pageSize, 4, 0)

);
$controls = array($pageControl);

ldap_set_option($l, LDAP_OPT_PROTOCOL_VERSION, 3);
ldap_bind($l, 'CN=bind-user,OU=my-users,DC=mydomain,DC=com', 'bind-user-password');

$continue = true;
while ($continue) {
    ldap_set_option($l, LDAP_OPT_SERVER_CONTROLS, $controls);
    $sr = ldap_search($l, 'OU=some-ou,DC=mydomain,DC=com', 'cn=*', array('sAMAccountName'), null, null, null, null);
    // there's the rub
    ldap_parse_result ($l, $sr, $errcode, $matcheddn, $errmsg, $referrals, $serverctrls);
    if (isset($serverctrls)) {
        foreach ($serverctrls as $i) {
            if ($i["oid"] == '1.2.840.113556.1.4.319') {
                    $i["value"]{8}   = chr($pageSize);
                    $i["iscritical"] = true;
                    $controls        = array($i);
                    break;
            }
        }
    }

    $info = ldap_get_entries($l, $sr);
    if ($info["count"] < $pageSize) {
        $continue = false;
    }

    for ($entry = ldap_first_entry($l, $sr); $entry != false; $entry = ldap_next_entry($l, $entry)) {
        $dn = ldap_get_dn($l, $entry);
    }
}

As you see, the only option to make all this work, is to mess with the ext/ldap source code and compile your own extension. Iñaki Arenaza provides several patches that can be applied to the PHP source to make patching a lot easier. The patches can be found here (last one for PHP 5.2.10 from June 24^th 2009) and there is an accompanying blog post available. Iñaki Arenaza even opened an issue in the PHP bug tracker on September 13^th 2005 offering his help – but there has been no reaction from the developer’s side. What a great pity.

So, if you have to use paged result sets in an Active Directory environment from within a PHP application you can choose between:

• patch your ext/ldap and compile your own extension as described in this article
• raise the MaxPageSize limit in your Active Directory server
• use a completely different approach bypassing ext/ldap and make use of the appropriate COM components (ADODB) as described here (this only works on Windows machines)

It could have been so easy, if…, yes if only the PHP developers considered applying the available patch to the ext/ldap source code.