Reading paged LDAP results with PHP is a show-stopper

November 6, 2009 at 16:52 10 comments

I was writing the schema introspection code for Zend_Ldap when I came around a problem with Active Directory’s MaxPageSize restriction. By default Active Directory allows only 1000 items to be returned on a single query, a number which is easily exceeded when reading an Active Directory’s classes and especially attributes from the schema tree. OK – one option would be to increase the MaxPageSize variable, but as the component should be usable on every Active Directory server I couldn’t go for that.

The second option that seemed possible makes use of the paged result sets that Active Directory returns on a query. This way led me into the world of LDAP server controls and deep into the ext/ldap source code. There is astonishingly little information on the topic of paged result sets and LDAP server controls in respect of PHP and ext/ldap. To be honest I assume that only one person really looked into this area seriously and even came up with a solution: Iñaki Arenaza (Blog, Twitter, Facebook, LinkedIn). His information provided here is the foundation of this article – the discoveries are absolutely not my work, they are all based on what Iñaki Arenaza dug out. I just wanted to bring a little light into this very specific topic (and summarize what I’ve answered on stackoverflow.com).

To make it short right from the beginning: it’s currently not possible to use paged results from an Active Directory with an unpatched PHP (ext/ldap).

Let’s take a closer look at what’s happening.

Active Directory uses a server control to accomplish server-side result paging. This control is described in RFC 2696 “LDAP Control Extension for Simple Paged Results Manipulation” . LDAP controls, which come in the flavors “server” and “client”, are extensions to the LDAP protocol to provide enhancements – result paging is one example, password policy is another one. Generally ext/ldap offers an access to LDAP control extensions via its ldap_set_option() and theLDAP_OPT_SERVER_CONTROLS and LDAP_OPT_CLIENT_CONTROLS option respectively. To setup the paged control we do need the control-oid, which is 1.2.840.113556.1.4.319, and we need to know how to encode the control-value (this is described in the RFC). The value is an octet string wrapping the BER-encoded version of the following SEQUENCE (copied from the RFC):

realSearchControlValue ::= SEQUENCE {
    size    INTEGER (0..maxInt),
                     -- requested page size from client
                     -- result set size estimate from server
    cookie  OCTET STRING
}

So we can setup the control prior to executing the LDAP desired query:

$pageSize    = 100;
$pageControl = array(
    // the control-oid
    'oid'        => '1.2.840.113556.1.4.319',
    // the operation should fail if the server is not able to support this control
    'iscritical' => true,
    // the required BER-encoded control-value
    'value'      => sprintf ("%c%c%c%c%c%c%c", 48, 5, 2, 1, $pageSize, 4, 0)
);

This allows us to send a paged query to the LDAP/AD server. But how do we know if there are more pages to follow and how do we have to send a query to get the next page of our result set?

The server responds with a result set that includes the required paging information – but PHP lacks a method to retrieve exactly this information from the result set. In fact ext/ldap provides the required function (ldap_parse_result()) but it fails to expose the required seventh and last argument serverctrlsp from the C function ldap_parse_result() in the LDAP API, which contains exactly the information we need to requery for consecutive pages. If we had this argument available in our PHP code, using paged controls would be straight forward:

$l = ldap_connect('somehost.mydomain.com');
$pageSize    = 100;
$pageControl = array(
    'oid'        => '1.2.840.113556.1.4.319',
    'iscritical' => true,
    'value'      => sprintf ("%c%c%c%c%c%c%c", 48, 5, 2, 1, $pageSize, 4, 0)

);
$controls = array($pageControl);

ldap_set_option($l, LDAP_OPT_PROTOCOL_VERSION, 3);
ldap_bind($l, 'CN=bind-user,OU=my-users,DC=mydomain,DC=com', 'bind-user-password');

$continue = true;
while ($continue) {
    ldap_set_option($l, LDAP_OPT_SERVER_CONTROLS, $controls);
    $sr = ldap_search($l, 'OU=some-ou,DC=mydomain,DC=com', 'cn=*', array('sAMAccountName'), null, null, null, null);
    // there's the rub
    ldap_parse_result ($l, $sr, $errcode, $matcheddn, $errmsg, $referrals, $serverctrls);
    if (isset($serverctrls)) {
        foreach ($serverctrls as $i) {
            if ($i["oid"] == '1.2.840.113556.1.4.319') {
                    $i["value"]{8}   = chr($pageSize);
                    $i["iscritical"] = true;
                    $controls        = array($i);
                    break;
            }
        }
    }

    $info = ldap_get_entries($l, $sr);
    if ($info["count"] < $pageSize) {
        $continue = false;
    }

    for ($entry = ldap_first_entry($l, $sr); $entry != false; $entry = ldap_next_entry($l, $entry)) {
        $dn = ldap_get_dn($l, $entry);
    }
}

As you see, the only option to make all this work, is to mess with the ext/ldap source code and compile your own extension. Iñaki Arenaza provides several patches that can be applied to the PHP source to make patching a lot easier. The patches can be found here (last one for PHP 5.2.10 from June 24th 2009) and there is an accompanying blog post available. Iñaki Arenaza even opened an issue in the PHP bug tracker on September 13th 2005 offering his help – but there has been no reaction from the developer’s side. What a great pity.

So, if you have to use paged result sets in an Active Directory environment from within a PHP application you can choose between:

  • patch your ext/ldap and compile your own extension as described in this article
  • raise the MaxPageSize limit in your Active Directory server
  • use a completely different approach bypassing ext/ldap and make use of the appropriate COM components (ADODB) as described here (this only works on Windows machines)

It could have been so easy, if…, yes if only the PHP developers considered applying the available patch to the ext/ldap source code.

About these ads

Entry filed under: PHP. Tags: , .

Remove nodes in SimpleXMLElement

10 Comments Add your own

  • 1. Igor  |  December 22, 2009 at 16:35

    Nice article… struggling with this issue an entire day.
    ======
    RE “The server responds with a result set that includes the required paging information…”
    It is possible to emulate this required pagination info?
    Can you post or send me via email print_r($i) when $i["oid"] == ’1.2.840.113556.1.4.319′, just to see how it looks like.
    ======
    Thanks!
    Igor.

    Reply
  • 2. PHP LDAP returning only some results  |  December 23, 2009 at 10:51

    [...] Did you find any solution? See Reading paged LDAP results with PHP is a show-stopper <?php. Seems this is the only article that shed light on that problem. Hopefully the answers on my [...]

    Reply
  • 3. S.A.  |  August 5, 2010 at 01:19

    Hello,

    Am planning to give this a try. Do you have any plans to post the patches for the latest php release (5.3.3)?

    Thanks

    Reply
    • 4. Stefan Gehrig  |  August 12, 2010 at 08:39

      Sorry, but I’m not really an expert in C and the PHP source code.
      Actually I don’t think that I am competent enough in programming PHP extensions to propose a solid patch.

      Reply
  • 5. Iñaki Arenaza  |  August 12, 2010 at 09:21

    Hi Stefan,

    thanks a lot for bringing some spot light on this issue :-)

    I just wanted to comment that the original proposed solution, patch and PHP bug are outdated by a more complete and easier to use solution in PHP bug #42060 ( http://bugs.php.net/bug.php?id=42060 ). There’s no need to deal with OIDs, BER encoding/decoding and so on.

    The patch has been applied to trunk already, but there are a couple of quirks that the PHP developers are waiting to be dealt with. I hope that once they get resolved, the patch is applied to trunk in an official way and merged into next stable release.

    Saludos.
    Iñaki.

    Reply
  • 6. Sébastien Cramatte  |  October 14, 2010 at 13:03

    Hi Stefan,

    Any chance to implement LDAP paging support into Zend_Ldap ?

    Thank you

    Reply
    • 7. Stefan Gehrig  |  October 14, 2010 at 13:08

      No – not until the underlying ext/ldap will support paging if you’re talking about LDAP server-side-paging. LDAP server-side-paging requires the mentioned low-level functionality that currently isn’t available in PHP.

      You can still page LDAP results with PHP using e.h. a LimitIterator or some small interface to the Zend_Paginator.

      Best regards

      Stefan

      Reply
      • 8. Sébastien Cramatte  |  October 14, 2010 at 13:29

        Thank you for your answer. It seems that server side paging has been merged to trunk in php 5.3 …

  • 9. David  |  April 29, 2011 at 01:33

    Hi Stefan, (and anyone else out there who lands here)

    I was unable to get the patches mentioned above to work for 5.3, so I submitted my own based off an existing one for the PHP 42060 bug report. It’s at http://bugs.php.net/patch-display.php?bug_id=42060&patch=paged-ldap-5.3&revision=latest along with brief instructions on how to get it set up.

    As I posted there, I’m NOT a C developer by trade, so any *helpful* criticism is most welcome. I’m hoping that by throwing this out there again, we can get the PHP folk to bump up the priority on this a bit more…

    Great post by the way – it was *very* helpful for me – so thank you.

    I hope this will work for someone else as well. The lack of paged LDAP responses in PHP had been a big obstacle in our institution’s moving away from Open LDAP to Active Directory.

    Reply
  • 10. John Barclay  |  March 31, 2012 at 06:57

    This is commited in php 5.4.

    See:

    http://us3.php.net/manual/en/function.ldap-control-paged-result-response.php

    http://us3.php.net/manual/en/function.ldap-control-paged-result.php

    The documentation is working its way through the issue queue. Not sure how to see the most current version, but here’s one: http://www.mail-archive.com/phpdoc@lists.php.net/msg46041.html

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Current articles

Twitter

del.icio.us

Certification


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: