headers
Dane Springmeyer | 22 Jul 2010 01:27
Gravatar

Optimizations around Mapnik's projection code

Recently I noticed the large number of:

#ifdef MAPNIK_THREADSAFE
    mutex::scoped_lock lock(mutex_);
#endif

wrapping the proj4 code in proj_transform.cpp and projections.cpp.

Proj4 has thread safety issues so this makes sense:

1) Projection initialization in not thread safe (when creating from epsg code)
2) Usage of prj_errno is not thread safe (error code pointer)
3) in rarer cases pj_transform is also not threadsafe (when transforming coordinates)

My thoughts were:

#1 As of proj 4.7 locking is done by proj4, so no need to duplicate locking in mapnik as far as I can tell

#2 is not a problem currently for mapnik because we don't attempt to do anything with errors (but we may in the
future - http://trac.mapnik.org/wiki/BoundsClipping)

#3 - can't do much about this.

So, I set out to deal with #1 and noticed a few more things:

1) there is a lock that affects proj_transform, even if we exit early because we don't need to reproject
(source == dest). What this means is that even when proj4 is not used, since we call proj_transform we're
hitting a lock for every coordinate reprojected
(http://trac.mapnik.org/browser/trunk/src/proj_transform.cpp?rev=2062#L74). Ouch.
(Continue reading)

Permalink | Reply |
headers
Tom Hughes | 22 Jul 2010 01:42
Picon
Gravatar

Re: Optimizations around Mapnik's projection code

On 22/07/10 00:27, Dane Springmeyer wrote:

> Recently I noticed the large number of:
>
> #ifdef MAPNIK_THREADSAFE
>      mutex::scoped_lock lock(mutex_);
> #endif
>
> wrapping the proj4 code in proj_transform.cpp and projections.cpp.
>
> Proj4 has thread safety issues so this makes sense:
>
> 1) Projection initialization in not thread safe (when creating from epsg code)
> 2) Usage of prj_errno is not thread safe (error code pointer)
> 3) in rarer cases pj_transform is also not threadsafe (when transforming coordinates)

I believe the locks were put in by me after Jon made mod_tile's renderd 
multi-threaded and found it was unstable.

All the proj4 doco I could find at the time just said it wasn't 
threadsafe so I wrapped everything in that mutex to make sure we
didn't reenter it at all.

If the issues are better understood now and we can improve things then 
that sounds good to me.

Tom

--

-- 
Tom Hughes (tom@...)
(Continue reading)

Permalink | Reply |
headers
Dane Springmeyer | 22 Jul 2010 23:11
Gravatar

Re: Optimizations around Mapnik's projection code


On Jul 21, 2010, at 4:42 PM, Tom Hughes wrote:

> On 22/07/10 00:27, Dane Springmeyer wrote:
> 
>> Recently I noticed the large number of:
>> 
>> #ifdef MAPNIK_THREADSAFE
>>     mutex::scoped_lock lock(mutex_);
>> #endif
>> 
>> wrapping the proj4 code in proj_transform.cpp and projections.cpp.
>> 
>> Proj4 has thread safety issues so this makes sense:
>> 
>> 1) Projection initialization in not thread safe (when creating from epsg code)
>> 2) Usage of prj_errno is not thread safe (error code pointer)
>> 3) in rarer cases pj_transform is also not threadsafe (when transforming coordinates)
> 
> I believe the locks were put in by me after Jon made mod_tile's renderd multi-threaded and found it was unstable.
> 
> All the proj4 doco I could find at the time just said it wasn't threadsafe so I wrapped everything in that
mutex to make sure we
> didn't reenter it at all.

Yes, I looks like there has been some good attention recently based on the wiki page at: http://trac.osgeo.org/proj/wiki/ThreadSafety

So, improvements in 4.7 (latest release) and likely also in the next release.

>
(Continue reading)

Permalink | Reply |

Gmane