Using nss-pam-ldapd in a large environment: Equivalent to 'nss_getgrent_skipmembers yes' to avoid unnecessary lookup flood

[Thomas Orgis <thomas.orgis [at] uni-hamburg.de>]

Hi,

we have a rather large LDAP setup with >60000 accounts. This has tranditionally 
mandated the use of 

nss_getgrent_skipmembers yes

in ldap.conf to have p.ex.

shell$ id $USER

not put out an excessive amount of LDAP queries while checking all the
group members. This basically disables listing of members of a group,
while lookups of the kind "Is $USER in $GROUP?" and "In which groups is
$USER?" still work.

This has been the working setup here for many years, with the old-style
nss_ldap.

Now, I'm setting up some CentOS 7 systems and see that nslcd produces
over 17000 queries to the LDAP server for one run of

shell$ id $USER

Reason: $USER is in a handful of groups, one of which amounts to about
17000 members. The query to get the list of groups is quickly answered,
but subsequently, nslcd goes on to check each group for its members,
looking up if they are posix accounts, probably, match uid, whatever.

This request-fest needs at least 13.5 s here, which is unacceptable.
Even the second run, with apparently most information cached in nslcd,
still needs over a whole second. There, only 20 LDAP requests are made
and considerable time is apparently spent working on the internal
cache.

There doesn't seem to be a clean solution to this, as the API layers
probably don't permit "skip looking at group members in this case as we
are not interested in them". That's why the existing solution to
nss_ldap was to just omit the group member lists. The only thing that
breaks is "list members of groups", which does not matter in practice
for us. There already isn't a "list all groups" as the LDAP server has
a limit on the number of replies, for performance and privacy reasons.

Some things just don't work when the number of users gets really large.

Is there a specific reason why this feature is not included in the
redesign?

When searching around about this topic, I find discussions relating to
sssd, which does not look like a real solution for me, as it just
reduces the time wasted by doing the unnecessary communication locally
instead of just not doing it.

I consider the one second it takes for `id` to return even with nearly
everthing being locally cached already too much. This can slow down
lots of processes, especially since in my application, it's usually
non-interactive batch jobs that need user information.

I'm not wholly sure if this is the correct patch, but it looks like it:

https://github.com/BinetReseau/BR_overlay/blob/master/sys-auth/nss_ldap/files/nss_ldap-254-nss_getgrent_skipmembers.patch

Some discussion on the nss_ldap release that included this:

http://osdir.com/ml/ldap.padl.nss/2007-10/msg00001.html

Can something like that be added to nss-pam-ldapd (not sure which
components need to be modified)? Not sure how this releates to
initgroups, I'm no NSS/LDAP expert, just a server admin (in this
situation;-).


Regards,

Thomas

-- 
Dr. Thomas Orgis
Universität Hamburg
RRZ / Zentrale Dienste / HPC
Schlüterstr. 70
20146 Hamburg
Tel.: 040/42838 8826
Fax: 040/428 38 6270

Attachment: smime.p7s
Description: S/MIME cryptographic signature

-- 
To unsubscribe send an email to
nss-pam-ldapd-users-unsubscribe@lists.arthurdejong.org or see
http://lists.arthurdejong.org/nss-pam-ldapd-users/