Skip to content

ldap no longer respects TLS_CACERT from ldaprc in ldap_start_tls() #18529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rainerjung opened this issue May 9, 2025 · 15 comments
Open

ldap no longer respects TLS_CACERT from ldaprc in ldap_start_tls() #18529

rainerjung opened this issue May 9, 2025 · 15 comments

Comments

@rainerjung
Copy link

Description

The following code:

<?php
ldap_set_option(NULL, LDAP_OPT_DEBUG_LEVEL, 7);
$connection = ldap_connect('ldap://ldap.mydomain.mytld');
ldap_start_tls($connection);
?>

using an ldaprc file like

TLS_REQCERT    demand
TLS_REQSAN    demand
TLS_CACERT    /path/to/my/ca-bundle.crt

Resulted in this output (using 8.4.7):

...
TLS certificate verification: depth: 2, err: 19, subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2
TLS certificate verification: Error, self-signed certificate in certificate chain
TLS trace: SSL3 alert write:fatal:unknown CA
TLS trace: SSL_connect:error in error
TLS: can't connect: error:0A000086:SSL routines::certificate verify failed (self-signed certificate in certificate chain).
ldap_err2string

Warning: ldap_start_tls(): Unable to start TLS: Connect error in /var/tmp/ldap.php on line 7
...

But I expected this output instead (using 8.4.6):

...
TLS certificate verification: depth: 2, err: 0, subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2
TLS certificate verification: depth: 1, err: 0, subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=GeoTrust TLS RSA CA G1, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2
TLS certificate verification: depth: 0, err: 0, subject: /C=DE/ST=Nordrhein-Westfalen/L=Bonn/O=kippdata informationstechnologie GmbH/CN=*.kippdata.de, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=GeoTrust TLS RSA CA G1
TLS trace: SSL_connect:SSLv3/TLS read server certificate
TLS trace: SSL_connect:TLSv1.3 read server certificate verify
TLS trace: SSL_connect:SSLv3/TLS read finished
TLS trace: SSL_connect:SSLv3/TLS write change cipher spec
TLS trace: SSL_connect:SSLv3/TLS write client certificate
TLS trace: SSL_connect:SSLv3/TLS write finished
...

The connection works with 8.4.7, if I add

ldap_set_option($connection, LDAP_OPT_X_TLS_CACERTFILE, "/path/to/my/ca-bundle.crt");

but it fails for 8.4.6, if I use to define the bundle that way and remove it from ldaprc.

So 8.4.6 need to have it in ldaprc, 8.4.7 needs to have it set via LDAP_OPT_X_TLS_CACERTFILE.

The problem shows up when using start_tls. When I connect directly to an ldaps URL, the bundle from ldaprc is used in 8.4.6 and 8.4.7 and the connection succeeds.

I guess this is related to #17776 and commit 389de7c .

PHP Version

PHP 8.4.7 (cli) (built: May  7 2025 06:16:13) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.4.7, Copyright (c) Zend Technologies

LDAP library OpenLDAP 2.6.9 with OpenSSL 3.4.1.

Operating System

RHEL 9

@remicollet
Copy link
Member

ldap_set_option not working in 8.4.6 is a real problem, so 8.4.7 is better for this.
ldaprc not working "only" for starttls is strange... need to investigate

@remicollet
Copy link
Member

remicollet commented May 10, 2025

BTW, I cannot reproduce, using PHP 8.3.21 or 8.4.7 (on Fedora 41 with openldap 2.6.9 or RHEL 9.5 with openldap 2.6.6 / openssl 3.2.2 from official RH channels, PHP from my repo)

Using a ~/.ldaprc file

With TLS_REQCERT=never and TLS_REQSAN=never
start_tls with no options works

----- Test STARTTLS with no options -----
TLS certificate verification: Error, unable to get local issuer certificate
LDAP bind succeeded

With TLS_REQCERT=demand and TLS_REQSAN=demand
start_tls with no options fails

----- Test STARTTLS with no options -----
TLS certificate verification: Error, unable to get local issuer certificate
TLS: can't connect: error:0A000086:SSL routines::certificate verify failed (unable to get local issuer certificate).
PHP Warning:  ldap_start_tls(): Unable to start TLS: Connect error in /work/build/phpmaster/bugldap.php on line 69
PHP Warning:  ldap_bind(): Unable to bind to server: Can't contact LDAP server in /work/build/phpmaster/bugldap.php on line 70
LDAP bind failed

@rainerjung
Copy link
Author

Sure, but the whole point of this ticket is, that verifying a certificate chain during start_tls fails although it is valid and the needed CA certificate is provided. When you disable the verification the test is no longer valid for this.

@bukka
Copy link
Member

bukka commented May 10, 2025

I'm wondering if it could be because ldaprc creates a global context which is now getting reset before the first use and it gets lost. If it's the case, it might be a bit tricky to get fixed. We would probably need to try to parse ldaprc ourselves (unless there is some API way to do that) or we could get the LDAP_OPT_X_TLS_SSL_CTX but there can be different backends so we would need to do some extra checks that OpenSSL is used and it's SSL* but that might get quite messy as we would also need to reconstruct chain and store it to some temp file so that's not probablly the way forward.

@bukka
Copy link
Member

bukka commented May 10, 2025

I just had a bit deeper look to the docs and code and it looks like it's really getting reset.

This is all actually quite well visible in https://github.com/openldap/openldap/blob/e460f00874352c3d2cdbca8a00a4112c7bc633ab/libraries/libldap/init.c . The core initialization function is ldap_int_initialize which is called from ldap_create (function called by ldap_initialiaze and friends) on the first initialization (based on global options check for ldo_valid that is set after the first initialization). The ldap_int_initialize includes the initialization for ldap.conf and ldaprc which is done in openldap_ldap_init_w_conf. It parses the config and sets the global options including the TLS ones.

The problem is that LDAP_OPT_X_TLS_NEWCTX resets the config so the global options from ldaprc are lost. I don't see any API way how re-init this conf loading so we would need to integrate ourselves which I'm not sure is a good idea.

So unless I missed something and my analysis of the code is incorrect, I think it might be best to to revert 389de7c and just add LDAP_OPT_X_TLS_NEWCTX constant to the master so users can use it if they want. We should, of course, document that this will also reset config in ldaprc.

@remicollet
Copy link
Member

Sure, but the whole point of this ticket is, that verifying a certificate chain during start_tls fails although it is valid and the needed CA certificate is provided. When you disable the verification the test is no longer valid for this.

The goal of this test is to ensure values from "ldaprc" are properly used.
And it works.

@rainerjung
Copy link
Author

rainerjung commented May 11, 2025

I misunderstood your comment and thought you wanted to say "no problem", but now I got it.

@rainerjung
Copy link
Author

It would be good to document under which circumstances values from ldaprc are picked up respectively get reset to default values.
I expected that ldaprc values are always respected unless you overwrite them individually in your PHP code. Since the recent PHP code change this expectation is no longer satisfied.
Instead it seems, that overwriting some values from ones PHP app resets other values.
I didn't dig into the code, whether that only happens when using ldap_start_tls() or also under other circumstances.

@bukka
Copy link
Member

bukka commented May 11, 2025

I did a bit more checking and the global options are really preserved. In this case it's kept in ldo_tls_cacertfile. So it should be used when the global context is re-set (at the beginning of the request).

Previous behaviour was that as soon as the context (SSL *) created, it was cached and couldn't be changed. This was also kept for all requests that use the same worker. It means for example when FPM (or apache prefork) child started and got the first request that initializes / starts tls session, it created a TLS context that was cached for all subsequent requests and it was not possible to change those values using ldap_set_option.

So there are few options what can be happening:

  • some of the previous requests set LDAP_OPT_X_TLS_CACERTFILE globally (NULL connection) which reset this value
  • the certificate got loaded but original path no longer exists (this should probably show warning though)
  • there is something else that resets ldo_tls_cacertfile - from the code it gets changed only in ldap_pvt_tls_set_option and ldap_int_tls_destroy though.

@rainerjung it would be great if you could do a bit more checking how this gets resets. If you can run it under gdb, it might be useful to add a break point to ldap_pvt_tls_set_option where LDAP_OPT_X_TLS_CACERTFILE to see where it gets set. Or maybe check the application what it exactly do.

@rainerjung
Copy link
Author

Thanks for the details to look for. gdb show four calls to ldap_pvt_tls_set_option(). The first three come from the three functional lines in ldaprc and set TLS_REQCERT, TLS_REQSAN and TLS_CACERT correctly as given in ldaprc. The stacks for these are mostly the same, namely

#0  ldap_pvt_tls_set_option (ld=ld@entry=0x0, option=option@entry=24578, arg=arg@entry=0x7fffffff9eae) at tls2.c:861
#1  0x00007ffff72aac28 in ldap_pvt_tls_config (ld=ld@entry=0x0, option=option@entry=24578, 
#2  0x00007ffff729d61c in ldap_int_conf_option (cmd=cmd@entry=0x7fffffff9ea0 "TLS_CACERT", 
#3  0x00007ffff729d8ab in openldap_ldap_init_w_conf (file=file@entry=0x7ffff72bd169 "ldaprc", userconf=userconf@entry=1) at init.c:346
#4  0x00007ffff729d97a in openldap_ldap_init_w_userconf (file=file@entry=0x7ffff72bd169 "ldaprc") at init.c:394
#5  0x00007ffff729dca7 in openldap_ldap_init_w_userconf (file=0x7ffff72bd169 "ldaprc") at init.c:751
#6  ldap_int_initialize (gopts=0x7ffff72cca40 <ldap_int_global_options>, dbglvl=<optimized out>) at init.c:748
#7  0x00007ffff7282d6a in ldap_create (ldp=ldp@entry=0x7fffffffa7b8) at open.c:132
#8  0x00007ffff7283010 in ldap_initialize (ldp=ldp@entry=0x7fffffffa808, url=url@entry=0x7ffff6e572b8 "ldap://ldap1.kippdata.de") at open.c:301
#9  0x00007ffff7e2aeff in zif_ldap_connect (execute_data=<optimized out>, return_value=0x7ffff6e14080)
#10 0x00007ffff78f0cc7 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER ()
#11 execute_ex (ex=0x0) at /path/to/my/php84/Zend/zend_vm_execute.h:58818
#12 0x00007ffff78ec445 in zend_execute (op_array=op_array@entry=0x7ffff6e91000, return_value=return_value@entry=0x0)
#13 0x00007ffff794fd00 in zend_execute_script (type=type@entry=8, retval=retval@entry=0x0, file_handle=file_handle@entry=0x7fffffffcdc0)
#14 0x00007ffff77f6546 in php_execute_script_ex (primary_file=<optimized out>, retval=<optimized out>)
#15 0x00000000006026d7 in do_cli (argc=6, argv=0xc02260) at /path/to/my/php84/sapi/cli/php_cli.c:936
#16 0x0000000000600e3c in main (argc=6, argv=0xc02260) at /path/to/my/php84/sapi/cli/php_cli.c:1311

Then there's exactly one additional call to ldap_pvt_tls_set_option(). It comes from zif_ldap_start_tls():

Breakpoint 1, ldap_pvt_tls_set_option (ld=ld@entry=0xd63190, option=option@entry=24591, arg=arg@entry=0x7fffffffa814) at tls2.c:861
861	in tls2.c
(gdb) bt full
#0  ldap_pvt_tls_set_option (ld=ld@entry=0xd63190, option=option@entry=24591, arg=arg@entry=0x7fffffffa814) at tls2.c:861
        lo = <optimized out>
        __PRETTY_FUNCTION__ = "ldap_pvt_tls_set_option"
#1  0x00007ffff729e8c1 in ldap_set_option (ld=0xd63190, option=option@entry=24591, invalue=invalue@entry=0x7fffffffa814) at options.c:860
        lo = 0xd63220
        dbglvl = 0x0
        rc = -1
        __PRETTY_FUNCTION__ = "ldap_set_option"
#2  0x00007ffff7e2e9af in zif_ldap_start_tls (execute_data=<optimized out>, return_value=0x7fffffffa880)
    at /path/to/my/php84/ext/ldap/ldap.c:3751
        link = 0x7ffff6e14100
        ld = 0x7ffff6e700c0
        rc = <optimized out>
        protocol = 3
        val = 0
#3  0x00007ffff78f1b6d in ZEND_DO_ICALL_SPEC_RETVAL_UNUSED_HANDLER ()
    at /path/to/my/php84/Zend/zend_vm_execute.h:1287
        call = 0x7ffff6e140b0
        fbc = 0x0
        ret = <optimized out>
        retval = {value = {lval = 8, dval = 3.9525251667299724e-323, counted = 0x8, str = 0x8, arr = 0x8, obj = 0x8, res = 0x8, ref = 0x8, ast = 0x8, zv = 0x8, 
            ptr = 0x8, ce = 0x8, func = 0x8, ww = {w1 = 8, w2 = 0}}, u1 = {type_info = 1, v = {type = 1 '\001', type_flags = 0 '\000', u = {extra = 0}}}, u2 = {
            next = 0, cache_slot = 0, opline_num = 0, lineno = 0, num_args = 0, fe_pos = 0, fe_iter_idx = 0, guard = 0, constant_flags = 0, extra = 0}}
        call_info = 4158753568
        call = <optimized out>
        fbc = <optimized out>
        ret = <optimized out>
        retval = {value = {lval = <optimized out>, dval = <optimized out>, counted = <optimized out>, str = <optimized out>, arr = <optimized out>, 
            obj = <optimized out>, res = <optimized out>, ref = <optimized out>, ast = <optimized out>, zv = <optimized out>, ptr = <optimized out>, 
            ce = <optimized out>, func = <optimized out>, ww = {w1 = <optimized out>, w2 = <optimized out>}}, u1 = {type_info = <optimized out>, v = {
              type = <optimized out>, type_flags = <optimized out>, u = {extra = <optimized out>}}}, u2 = {next = <optimized out>, cache_slot = <optimized out>, 
            opline_num = <optimized out>, lineno = <optimized out>, num_args = <optimized out>, fe_pos = <optimized out>, fe_iter_idx = <optimized out>, 
            guard = <optimized out>, constant_flags = <optimized out>, extra = <optimized out>}}
        call_info = <optimized out>
#4  execute_ex (ex=0xd63190) at /path/to/my/php84/Zend/zend_vm_execute.h:58813
        vm_stack_data = {
          hybrid_jit_red_zone = '\000' <repeats 20 times>, "\377\177\000\000\000\000\000\000\000\000\000\000p@\341\366\377\177\000\000P\214\341\367\377\177\000", 
          orig_opline = 0xc02260, orig_execute_data = 0xbfa740 <OPTIONS>}

So the first three calls operate on the global LDAP config and the third one on the config with address 0xd63190.

This brings me to the following code in ldap.c, that was added by the commit in question:

1007 #ifdef LDAP_OPT_X_TLS_NEWCTX
1008                 if (LDAPG(tls_newctx) && url && !strncmp(url, "ldaps:", 6)) {
1009                         int val = 0;
1010 
1011                         /* ensure all pending TLS options are applied in a new context */
1012                         if (ldap_set_option(NULL, LDAP_OPT_X_TLS_NEWCTX, &val) != LDAP_OPT_SUCCESS) {
1013                                 zval_ptr_dtor(return_value);
1014                                 php_error_docref(NULL, E_WARNING, "Could not create new security context");
1015                                 RETURN_FALSE;
1016                         }
1017                         LDAPG(tls_newctx) = false;
1018                 }
1019 #endif

I have not yet understood that construct, but it comments "ensure all pending TLS options are applied in a new context" and runs only when an ldaps URL is used. In our case, we have an ldap url but use start_tls afterwards. So I think the code does not "apply all pending TLS options to the new context" and thus the CACERT option gets lost.

I can not judge, whether the code could and should also run for a plain ldap URL so that a followup start_tls call would succeed.

@rainerjung
Copy link
Author

Sorry, I see, that start_tls seems to do something similar:

3731 #ifdef HAVE_LDAP_START_TLS_S
3732 /* {{{ Start TLS */
3733 PHP_FUNCTION(ldap_start_tls)
3734 {
...
3749         if (((rc = ldap_set_option(ld->link, LDAP_OPT_PROTOCOL_VERSION, &protocol)) != LDAP_SUCCESS) ||
3750 #ifdef LDAP_OPT_X_TLS_NEWCTX
3751                 (LDAPG(tls_newctx) && (rc = ldap_set_option(ld->link, LDAP_OPT_X_TLS_NEWCTX, &val)) != LDAP_OPT_SUCCESS) ||
3752 #endif
3753                 ((rc = ldap_start_tls_s(ld->link, NULL, NULL)) != LDAP_SUCCESS)
3754         ) {
3755                 php_error_docref(NULL, E_WARNING,"Unable to start TLS: %s", ldap_err2string(rc));
3756                 RETURN_FALSE;
3757         }

I checked, that LDAP_OPT_X_TLS_NEWCTX is defined in my include/ldap.h:

define LDAP_OPT_X_TLS_NEWCTX		0x600f

(from openldap 2.6.9).
I will try to debug a bit more...

@bobvandevijver
Copy link

We're hit by this issue as well, where the TLS_CACERT from the system /etc/ssl/certs/ca-certificates.crt is no longer respected when using ldap://host with starttls.

Only by explicitly calling ldap_set_option($connection, LDAP_OPT_X_TLS_CACERTFILE, '/etc/ssl/certs/ca-certificates.crt') after ldap_connect and before ldap_start_tls we can resolve the issue, which is a challenge to do so due to how Symfony has defined their adapter:

https://github.com/symfony/symfony/blob/db8e84d74d58e6f47a66160a806990d6f00a5e96/src/Symfony/Component/Ldap/Adapter/ExtLdap/Connection.php#L33

For me, this setting should be kept for both context types.

@rainerjung
Copy link
Author

I have investigated the OpenLDAP source code. My understanding is as follows:

  1. TLS related config is placed in three areas which are handled differently:

a) direct items in ld->ld_options aka ld->ldc->ldc_options

Examples are the flags

    int                     ldo_tls_require_cert;
    int                     ldo_tls_require_san;

which determine if and how certificates get checked.

b) indirect items in ld->ld_options->ldo_tls_info aka ld->ldc->ldc_options->ldo_tls_info

Examples are:

    char            *lt_certfile;
    char            *lt_keyfile;
    char            *lt_cacertfile;
    char            *lt_cacertdir;
    char            *lt_ciphersuite;
    int             lt_protocol_min;
    int             lt_protocol_max;

Some macro abbreviations exist like eg.

ldo_tls_certfile for ldo_tls_info.lt_certfile

c) the SSL context in ld->ld_options->ldo_tls_ctx aka ld->ldc->ldc_options->ldo_tls_ctx

    void            *ldo_tls_ctx;

Implementation specific, eg. for OpenSSL etc.

How do the three parts behave with respect to inheriting settings from the global context to an LDAP instance?

First: once the SSL context c) is initialized, any change to settings in a) or b) will no longer influence that context. It gets created from a) and b) but from then on it is independent of them, except when it gets freshly created. We'll come back to this in a minute.

That means setting TLS config items after the SSL context got initialized does no longer change it, except if being recreated.

The first SSL context seems always to get initialized from the global settings.

When a non-global LDAP instance is created in ldap_create( LDAP **ldp ), eg. via ldap_initialize(LDAP **ldp, LDAP_CONST char *url), the config items in a) get copied from the global settings via memcpy. The settings in b) get reset (!) and the ssl ctx is NULL:

int
ldap_create( LDAP **ldp )
{
...
AC_MEMCPY(&ld->ld_options, gopts, sizeof(ld->ld_options));
...
memset( &ld->ld_options.ldo_tls_info, 0,
sizeof( ld->ld_options.ldo_tls_info ));
ld->ld_options.ldo_tls_ctx = NULL;
...
}

The SSL context gets initialized either during ldap_int_tls_connect(...). If it is not yet initialized it gets always initialized from the global settings.

Or it gets initialized or recreated via ldap_set_option for option LDAP_OPT_X_TLS_NEWCTX. In that case it gets initialized from a) and b) of the LDAP instance. Remember the LDAP instance always starts with b) being zeroed.

So how does that relate to the observed behavior in PHP land:

  • setting

TLS_REQCERT demand
TLS_REQSAN demand

in ldaprc goes into part a) in the global options and always gets inherited to any LDAP instance unless explicitly overwritten with ldap_set_option.

Setting TLS_CACERT in ldaprc goes into b). It ends up in the first SSL context, but whenever a new SSL context is created from and LDAP instance, because b) gets zeroed when creating the instance, it gets lost in the SSL context. The only way to create a new SSL context is ldap_set_option with LDAP_OPT_X_TLS_NEWCTX.

When one wants to overwrite global settings via ldap_set_option on an LDAP instance before the SSL context is created, one has to make sure, that the new SSL context is not simply the global one. IMHO therefore one needs to use LDAP_OPT_X_TLS_NEWCTX. But then all global b) configs are lost unless recreated with explicit calls to ldap_set_option.

I think the memset of ldo_tls_info to 0 in ldap_create() is not a good idea. Of course one would have to do a deep copy instead of just a memcpy in order to get an independent copy of strings in the created LDAP instead of a copied pointer. That way the new LDAP would inherit the configuration and be adjustable without also changing the global config.

As a result I would say there is no good solution. But at least one should document, that using start_tls destroys the global config for

    char            *lt_certfile;
    char            *lt_keyfile;
    char            *lt_dhfile;
    char            *lt_cacertfile;
    char            *lt_cacertdir;
    char            *lt_ciphersuite;
    char            *lt_crlfile;
    char            *lt_randfile;   /* OpenSSL only */
    char            *lt_ecname;             /* OpenSSL only */
    int             lt_protocol_min;
    int             lt_protocol_max;
    struct berval   lt_cacert;
    struct berval   lt_cert;
    struct berval   lt_key;

(the names in the global config ldaprc or env vbars differ, but you get the idea).

And one should decide, whether the change in behavior is OK for a patch release like 8.4.7 or not. I myself am undecided and fine with it each way.

@bobvandevijver
Copy link

Note that this is also an issue with PHP 8.3.21 (using the Sury Debian packages).

For me the only choice would be to keep the global settings when using start_tls. If you explicitly want a clean context you have the LDAP_OPT_X_TLS_NEWCTX available right?

@remicollet
Copy link
Member

remicollet commented May 13, 2025

@rainerjung thanks for the detailed diag.

@bobvandevijver no, LDAP_OPT_X_TLS_NEWCTX is not available from PHP

I really thinks this should be reported upstream (openldap), string options should be inherited like other.

A possible workaround is proposed in PR #18547 (draft for now)

Other eyes / tests / feedback welcome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants