gmane.comp.ldap.umich

Several DN one LDAP query

Hello,

I'd like to know if it's correct to retrieve several entries from a
directory in one LDAP query based on the DN.
I have several group DN:
cn=marketing,ou=Groups,dc=example,dc=com
cn=sales,ou=Groups,dc=example,dc=com
And I'd like to get the entries of all DN in only one query (I
actually want to get all the members of these groups).

Is it correct/possible to do this or do I have to run one query per DN ?

Thanks,
Manuel

headers

Hallvard B Furuseth | 6 Oct 16:28

Re: Several DN one LDAP query

Manuel Vacelet writes:
> I'd like to know if it's correct to retrieve several entries from a
> directory in one LDAP query based on the DN.
> I have several group DN:
> cn=marketing,ou=Groups,dc=example,dc=com
> cn=sales,ou=Groups,dc=example,dc=com
> And I'd like to get the entries of all DN in only one query (I
> actually want to get all the members of these groups).
> 
> Is it correct/possible to do this or do I have to run one query per DN ?

Normally one query per DN, since each LDAP operation has one baseDN.

Though it may be possible to hack it: Search with
  base   "ou=Groups,dc=example,dc=com"
  filter "(&(your intended filter)(|(cn:dn:=marketing)(cn:dn:=sales)))"

That finds entries matching your filter which also has "cn=marketing" or
"cn=sales" in the DN.  However that means it'd also match e.g. an entry
named cn=sales,cn=Somewhere,cn=Else,ou=Groups,dc=example,dc=com.

--

-- 
Hallvard

headers

Manuel Vacelet | 8 Oct 14:06

Re: Several DN one LDAP query

On Mon, Oct 6, 2008 at 4:28 PM, Hallvard B Furuseth
<h.b.furuseth@...> wrote:
> Manuel Vacelet writes:
>> I'd like to know if it's correct to retrieve several entries from a
>> directory in one LDAP query based on the DN.
>> I have several group DN:
>> cn=marketing,ou=Groups,dc=example,dc=com
>> cn=sales,ou=Groups,dc=example,dc=com
>> And I'd like to get the entries of all DN in only one query (I
>> actually want to get all the members of these groups).
>>
>> Is it correct/possible to do this or do I have to run one query per DN ?
>
> Normally one query per DN, since each LDAP operation has one baseDN.

Ok, thanks for the answer (thanks to Emmanuel who replied to me in private too).

> Though it may be possible to hack it: Search with
>  base   "ou=Groups,dc=example,dc=com"
>  filter "(&(your intended filter)(|(cn:dn:=marketing)(cn:dn:=sales)))"
>
> That finds entries matching your filter which also has "cn=marketing" or
> "cn=sales" in the DN.  However that means it'd also match e.g. an entry
> named cn=sales,cn=Somewhere,cn=Else,ou=Groups,dc=example,dc=com.

If limit the scope of my query to "One" this shouldn't happen isn't it ?

headers

Hallvard B Furuseth | 9 Oct 23:15

Re: Several DN one LDAP query

Manuel Vacelet writes:
>> Though it may be possible to hack it: Search with
>>  base   "ou=Groups,dc=example,dc=com"
>>  filter "(&(your intended filter)(|(cn:dn:=marketing)(cn:dn:=sales)))"
>>
>> That finds entries matching your filter which also has "cn=marketing" or
>> "cn=sales" in the DN.  However that means it'd also match e.g. an entry
>> named cn=sales,cn=Somewhere,cn=Else,ou=Groups,dc=example,dc=com.
> 
> If limit the scope of my query to "One" this shouldn't happen isn't it ?

Sorry, I seem to have "seen" another question than you asked
Yes, that's right.  Or since I don't think that search can make use of
indexes (though it depends on the implementation), just use the filter
    "(|(cn=marketing)(cn=sales))"
That could find groups like this too however:
    dn: cn=sneaky group,ou=Groups,dc=example,dc=com
    cn: sneaky group
    cn: marketing
    ...
If that's a problem but you want indexing you could combine the two: the
first for indexing, the second to filter out sneaky groups:
   (|(&(cn=marketing)(cn:dn:=marketing))(&(cn=sales)(cn:dn:=sales)))

Hopefully I got it right this time

--

-- 
Hallvard

(Continue reading)

headers

ELCIN HAKTANIR | 10 Oct 16:39

slapadd newbie

Question1:
---------------
Is it rational that slapadd took 31 minutes for 100,000 entries(23Kbyte per subscriber i guess) without index.?
I think it is so slow.isn't it?
What have i done wrong then?Could you please help to reduce this time ?
Question2:
--------------
And when 15 million subscribers is considered what should be the time to slapadd them?

Question3:
-----------------
i don't plan to make searches frequently,instead i will give exact DNs of the entries while the system is being used with 15 million subscribers.
Do i have to define indexes?
Thank you in advance.

Configuration information about my Environment:
---------------------------------------------------------------------
openldap-2.4.10-sol9-sparc-local.gz installed on a System with
Configuration: Sun Microsystems sun4u Sun Fire 280R (2 X UltraSPARC-III+)
System clock frequency: 150 MHz
Memory size: 2048 Megabytes

My slapadd command:
-------------------------------------
/usr/local/sbin/slapadd -l /usr/local/etc/openldap/ldifs/subscribersPart100.ldif -f /usr/local/etc/openldap/slapd.conf -b o=sdftest -d 256 -q

my DB_CONFIG file is:
---------------------------------------
set_cachesize 1 209715200 0
set_flags DB_LOG_AUTOREMOVE
#
# Setting set_tas_spins reduces resource contention from multiple clients on systems with multiple CPU's.
set_tas_spins 1
set_flags DB_TXN_NOSYNC
set_lg_max 536870912
set_lg_bsize 134217728
set_lg_dir /usr/local/var/openldap-logs
set_shm_key 1

my slapd.conf file is:
---------------------------------
database hdb
suffix "o=sdftest"
rootdn "cn=sdf,o=sdftest"
rootpw admin234
directory /usr/local/var/sdftest
index default eq
index objectClass
checkpoint 512 30
dirtyread

--------------------------------------

Bu elektronik posta ve onunla iletilen bütün dosyalar gizlidir sadece yukarıda isimleri belirtilen kişiler arasında özel haberleşme amacını taşımaktadır. Size yanlışlıkla ulaşmıssa bu elektonik postanın içeriğini açıklamanız , kopyalamanız, yönlendirmeniz ve kullanmanız kesinlikle yasaktır. Lütfen mesajı geri gönderiniz ve sisteminizden siliniz. Vodafone Teknoloji Hizmetleri A.Ş. bu mesajın içeriği ile ilgili olarak hiç bir hukuksal sorumluluğu kabul etmez.

This electonic mail and any files transmitted with it are intended for the private use of the persons named above. If you received this message in error, forwarding, copying or use of any of the information is strictly prohibited. Please immediately notify the sender and delete it from your system. Vodafone Teknoloji Hizmetleri A.S. does not accept legal responsibility for the contents of this message.
--------------------------------------

headers

Joao Amancio | 10 Oct 17:08

Re: slapadd newbie

I'm not an expert but want to suggest you to do two things:

Study and apply indexes for your own use
Utilize 'cachesize' in your "slapd.conf"

Again I think it would improve better performance.

Best regards,
João Ferreira

On Fri, Oct 10, 2008 at 11:39 AM, ELCIN HAKTANIR <elcin.haktanir <at> vodafone.com> wrote:

Question1:
---------------
Is it rational that slapadd took 31 minutes for 100,000 entries(23Kbyte per subscriber i guess) without index.?
I think it is so slow.isn't it?
What have i done wrong then?Could you please help to reduce this time ?
Question2:
--------------
And when 15 million subscribers is considered what should be the time to slapadd them?

Question3:
-----------------
i don't plan to make searches frequently,instead i will give exact DNs of the entries while the system is being used with 15 million subscribers.
Do i have to define indexes?
Thank you in advance.

Configuration information about my Environment:
---------------------------------------------------------------------
openldap-2.4.10-sol9-sparc-local.gz installed on a System with
Configuration: Sun Microsystems sun4u Sun Fire 280R (2 X UltraSPARC-III+)
System clock frequency: 150 MHz
Memory size: 2048 Megabytes

My slapadd command:
-------------------------------------
/usr/local/sbin/slapadd -l /usr/local/etc/openldap/ldifs/subscribersPart100.ldif -f /usr/local/etc/openldap/slapd.conf -b o=sdftest -d 256 -q

my DB_CONFIG file is:
---------------------------------------
set_cachesize 1 209715200 0
set_flags DB_LOG_AUTOREMOVE
#
# Setting set_tas_spins reduces resource contention from multiple clients on systems with multiple CPU's.
set_tas_spins 1
set_flags DB_TXN_NOSYNC
set_lg_max 536870912
set_lg_bsize 134217728
set_lg_dir /usr/local/var/openldap-logs
set_shm_key 1

my slapd.conf file is:
---------------------------------
database hdb
suffix "o=sdftest"
rootdn "cn=sdf,o=sdftest"
rootpw admin234
directory /usr/local/var/sdftest
index default eq
index objectClass
checkpoint 512 30
dirtyread

--------------------------------------

Bu elektronik posta ve onunla iletilen bütün dosyalar gizlidir sadece yukarıda isimleri belirtilen kişiler arasında özel haberleşme amacını taşımaktadır. Size yanlışlıkla ulaşmıssa bu elektonik postanın içeriğini açıklamanız , kopyalamanız, yönlendirmeniz ve kullanmanız kesinlikle yasaktır. Lütfen mesajı geri gönderiniz ve sisteminizden siliniz. Vodafone Teknoloji Hizmetleri A.Ş. bu mesajın içeriği ile ilgili olarak hiç bir hukuksal sorumluluğu kabul etmez.

This electonic mail and any files transmitted with it are intended for the private use of the persons named above. If you received this message in error, forwarding, copying or use of any of the information is strictly prohibited. Please immediately notify the sender and delete it from your system. Vodafone Teknoloji Hizmetleri A.S. does not accept legal responsibility for the contents of this message.
--------------------------------------

headers

Quanah Gibson-Mount | 10 Oct 17:10

Re: slapadd newbie

--On Friday, October 10, 2008 12:08 PM -0300 Joao Amancio 
<jjamancio@...> wrote:

>
> I'm not an expert but want to suggest you to do two things:
>
>
>   • Study and apply indexes for your own use
>
>   • Utilize 'cachesize' in your "slapd.conf"
> Again I think it would improve better performance.

cachesize isn't going to help much with slapadd.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

headers

Quanah Gibson-Mount | 10 Oct 17:09

Re: slapadd newbie

--On Friday, October 10, 2008 5:39 PM +0300 ELCIN HAKTANIR 
<elcin.haktanir@...> wrote:

>
> Question1:
> ---------------
> Is it rational that slapadd took 31 minutes for 100,000 entries(23Kbyte
> per subscriber i guess) without index.?
> I think it is so slow.isn't it?
> What have i done wrong then?Could you please help to reduce this time ?

Verify your DB_CONFIG file.  Set the tool-threads value in slapd.conf, 
assuming OpenLDAP 2.3 or later.  Don't use debug flags with slapadd.  Also 
the disk speed is going to have an impact.

> Question2:
> --------------
> And when 15 million subscribers is considered what should be the time to
> slapadd them?

Depends on the size of the entry, the amount of indexing you have done, and 
whether or not you've tuned your server for optimal performance.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

headers

Gavin Henry | 10 Oct 17:14

Re: slapadd newbie


----- "Quanah Gibson-Mount" <quanah@...> wrote:

> --On Friday, October 10, 2008 5:39 PM +0300 ELCIN HAKTANIR 
> <elcin.haktanir@...> wrote:
> 
> >
> > Question1:
> > ---------------
> > Is it rational that slapadd took 31 minutes for 100,000
> entries(23Kbyte
> > per subscriber i guess) without index.?
> > I think it is so slow.isn't it?
> > What have i done wrong then?Could you please help to reduce this
> time ?
> 
> Verify your DB_CONFIG file.  Set the tool-threads value in slapd.conf,
> 
> assuming OpenLDAP 2.3 or later.  Don't use debug flags with slapadd. 
> Also 
> the disk speed is going to have an impact.

Also see the -q flag:

-q     enable quick (fewer integrity checks) mode.  Does fewer consistency checks on the input data, and no
consistency checks when writing the database.  Improves the load time but if any errors or interruptions
occur the resulting database will be unusable.

--

-- 
Kind Regards,

Gavin Henry.

T +44 (0) 1224 279484
M +44 (0) 7930 323266
F +44 (0) 1224 824887
E ghenry@...

Open Source. Open Solutions(tm).

http://www.suretecsystems.com/

Suretec Systems is a limited company registered in Scotland. Registered
number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie,
Aberdeenshire, AB51 4FP.

headers

Quanah Gibson-Mount | 10 Oct 17:15

Re: slapadd newbie

--On Friday, October 10, 2008 4:14 PM +0100 Gavin Henry 
<ghenry@...> wrote:

>
> ----- "Quanah Gibson-Mount" <quanah@...> wrote:
>
>> --On Friday, October 10, 2008 5:39 PM +0300 ELCIN HAKTANIR
>> <elcin.haktanir@...> wrote:
>>
>> >
>> > Question1:
>> > ---------------
>> > Is it rational that slapadd took 31 minutes for 100,000
>> entries(23Kbyte
>> > per subscriber i guess) without index.?
>> > I think it is so slow.isn't it?
>> > What have i done wrong then?Could you please help to reduce this
>> time ?
>>
>> Verify your DB_CONFIG file.  Set the tool-threads value in slapd.conf,
>>
>> assuming OpenLDAP 2.3 or later.  Don't use debug flags with slapadd.
>> Also
>> the disk speed is going to have an impact.
>
> Also see the -q flag:
>
> -q     enable quick (fewer integrity checks) mode.  Does fewer
> consistency checks on the input data, and no consistency checks when
> writing the database.  Improves the load time but if any errors or
> interruptions occur the resulting database will be unusable.

His example shows he was already using it. ;)

/usr/local/sbin/slapadd -l 
/usr/local/etc/openldap/ldifs/subscribersPart100.ldif  -f 
/usr/local/etc/openldap/slapd.conf -b o=sdftest -d 256 -q

--Quanah

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

headers

Gavin Henry | 10 Oct 17:18

Re: slapadd newbie

> /usr/local/sbin/slapadd -l 
> /usr/local/etc/openldap/ldifs/subscribersPart100.ldif  -f 
> /usr/local/etc/openldap/slapd.conf -b o=sdftest -d 256 -q

Oops.

--

-- 
Kind Regards,

Gavin Henry.
OpenLDAP Engineering Team.

E ghenry@...

Community developed LDAP software.

http://www.openldap.org/project/

headers

ELCIN HAKTANIR | 10 Oct 17:22

Re: slapadd newbie

100,000,000 subscribers is:2.1G total.

bash-2.05# du -c -h *.bdb

157M dn2id.bdb
2.0G id2entry.bdb
2.0M objectClass.bdb
2.1G total

again my Configuration information about my Environment:
---------------------------------------------------------------------
openldap-2.4.10-sol9-sparc-local.gz installed on a System with Configuration:

bash-2.05# prtdiag
System Configuration: Sun Microsystems sun4u Sun Fire 280R (2 X UltraSPARC-III+)
System clock frequency: 150 MHz
Memory size: 2048 Megabytes

========================= CPUs ===============================================

Run E$ CPU CPU
Brd CPU MHz MB Impl. Mask
--- --- ---- ---- ------- ----
A 0 900 8.0 US-III+ 2.3
B 1 900 8.0 US-III+ 2.3

========================= Memory Configuration ===============================

Logical Logical Logical
MC Bank Bank Bank DIMM Interleave Interleaved
Brd ID num size Status Size Factor with
---- --- ---- ------ ----------- ------ ---------- -----------
CA 0 0 1024MB no_status 512MB 2-way 0
CA 0 2 1024MB no_status 512MB 2-way 0

Gavin Henry <ghenry <at> suretecsystems.com>
Sent by: bounce-ldap-5624112 <at> listserver.itd.umich.edu

10/10/2008 06:14 PM

Please respond to
Gavin Henry <ghenry-0iySFhgulYrkQYj/0HfcvtBPR1lH4CV8@public.gmane.org>

To	Quanah Gibson-Mount <quanah <at> zimbra.com>
cc	ELCIN HAKTANIR <elcin.haktanir <at> vodafone.com>, ldap-63aXycvo3TyHXe+LvDLADg@public.gmane.org
Subject	[ldap] Re: slapadd newbie

----- "Quanah Gibson-Mount" <quanah-zAQalKWTt5vQT0dZR+AlfA@public.gmane.org> wrote:

> --On Friday, October 10, 2008 5:39 PM +0300 ELCIN HAKTANIR
> <elcin.haktanir-ANTagKRnAhdWk0Htik3J/w@public.gmane.org> wrote:
>
> >
> > Question1:
> > ---------------
> > Is it rational that slapadd took 31 minutes for 100,000
> entries(23Kbyte
> > per subscriber i guess) without index.?
> > I think it is so slow.isn't it?
> > What have i done wrong then?Could you please help to reduce this
> time ?
>
> Verify your DB_CONFIG file. Set the tool-threads value in slapd.conf,
>
> assuming OpenLDAP 2.3 or later. Don't use debug flags with slapadd.
> Also
> the disk speed is going to have an impact.

Also see the -q flag:

-q enable quick (fewer integrity checks) mode. Does fewer consistency checks on the input data, and no consistency checks when writing the database. Improves the load time but if any errors or interruptions occur the resulting database will be unusable.

--
Kind Regards,

Gavin Henry.

T +44 (0) 1224 279484
M +44 (0) 7930 323266
F +44 (0) 1224 824887
E ghenry-0iySFhgulYrkQYj/0HfcvtBPR1lH4CV8@public.gmane.org

Open Source. Open Solutions(tm).

http://www.suretecsystems.com/

Suretec Systems is a limited company registered in Scotland. Registered
number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie,
Aberdeenshire, AB51 4FP.

--------------------------------------

Bu elektronik posta ve onunla iletilen bütün dosyalar gizlidir sadece yukarıda isimleri belirtilen kişiler arasında özel haberleşme amacını taşımaktadır. Size yanlışlıkla ulaşmıssa bu elektonik postanın içeriğini açıklamanız , kopyalamanız, yönlendirmeniz ve kullanmanız kesinlikle yasaktır. Lütfen mesajı geri gönderiniz ve sisteminizden siliniz. Vodafone Teknoloji Hizmetleri A.Ş. bu mesajın içeriği ile ilgili olarak hiç bir hukuksal sorumluluğu kabul etmez.

This electonic mail and any files transmitted with it are intended for the private use of the persons named above. If you received this message in error, forwarding, copying or use of any of the information is strictly prohibited. Please immediately notify the sender and delete it from your system. Vodafone Teknoloji Hizmetleri A.S. does not accept legal responsibility for the contents of this message.
--------------------------------------