#mapnik log: Sunday 12, April 2009

2009 | 04

previous | next
00:32:35 *** IvanSanchez (n=ivan@74.3.222.87.dynamic.jazztel.es) has joined #mapnik
00:51:45 <nikq> Mapnik Trac: DebianInstallation edited | http://trac.mapnik.org/wiki/DebianInstallation?version=3
00:53:38 <springmeyer> hmm, you'll note on the ubuntu page that I've listed all the libboost stuff
00:53:39 <springmeyer> http://trac.mapnik.org/wiki/UbuntuInstallation
00:53:56 <springmeyer> but last time on lenny it sure seemed like liboost-dev would grab everything
00:54:28 <springmeyer> IvanSanchez: we should add the whole list back to the debian one if that is not the case
00:54:50 <IvanSanchez> I've added a note at the bottom
00:55:08 <springmeyer> sure
00:55:14 <IvanSanchez> though spelling out the pacjages for the optional libraries would be nice
00:55:54 <springmeyer> or libboost*-dev :)
00:56:07 <IvanSanchez> :-D
00:56:32 <IvanSanchez> I didn't find any packages for ogr nor ociei at first sight, though :-/
00:56:48 <springmeyer> ogr comes with gdal
00:57:06 <springmeyer> ociei is oracle thus not open source or freely available
00:57:34 <springmeyer> are you getting Scons output complaining ogr is not found?
00:57:53 <IvanSanchez> aye
00:58:24 <IvanSanchez> I was missing libgdal-dev (though mapnik is compiling... so far, so good)
00:58:54 <springmeyer> ya, optional - only needed for the gdal or ogr plugins
00:59:12 <springmeyer> so in the past you've only compiled with autotools?
00:59:26 <IvanSanchez> this is my first time compiling mapnik O:-)
00:59:38 <springmeyer> oh oh
00:59:43 <IvanSanchez> and as an avid debian user, I'm used to apt-get everything
01:00:08 <IvanSanchez> today I'm battling against the evil mod_tile
01:00:23 <springmeyer> gocha, so you've been doing mapnik 0.5.1 packages
01:00:23 <IvanSanchez> oohh, it built
01:00:33 <nikq> Mapnik Trac: Ticket #309 (Extend filters to have a length test) updated | http://trac.mapnik.org/ticket/309#comment:2
01:00:39 <IvanSanchez> yeah, and mod_tile doesn't like single-threaded 0.5.1
01:00:50 <springmeyer> 0.6.0 is already on ticket away from unstable in lenny
01:00:55 <springmeyer> on/one
01:01:30 <IvanSanchez> oh, great
01:01:34 <springmeyer> ya, mod_tile, osm2pgsql, mapnik, osm styles.... they all play nice with svn :)
01:01:47 <Ldp__> which is the stack I'm using now
01:01:49 <Ldp__> all from svn
01:02:12 <IvanSanchez> whoa, renderd just built
01:02:24 <springmeyer> rock on!
01:02:25 * IvanSanchez goes off to reboot apache
01:03:55 <IvanSanchez> dang
01:04:04 <IvanSanchez> renderd crashed... again :-(
01:04:17 <Ldp__> did you make clean?
01:04:31 <IvanSanchez> though so... doing again just in case I missed it
01:06:15 <IvanSanchez> nah, keeps crashing
01:06:26 <IvanSanchez>   File "./renderd.py", line 254, in render_meta    m.resize(render_size, render_size)AttributeError: 'Map' object has no attribute 'resize'
01:06:44 <Ldp__> py?
01:06:52 <Ldp__> I'm using the compiled version
01:07:30 <springmeyer> ya, resize was added after 0.5.x series
01:07:32 <Ldp__> trying renderd.py now
01:07:51 <springmeyer> IvanSanchez: mod_tile must still be linking against old mapnik files
01:07:56 <IvanSanchez> uh... maybe I assumed I had to run the .py instead of the compiled
01:08:20 <springmeyer> oh, the py, duh
01:08:32 <springmeyer> ya, so that will be importing the python bindings of mapnik
01:08:59 <springmeyer> so you need to uninstall the mapnik 0.5.1 package
01:09:07 <Ldp__> renderd.py doesn't crash for me, but it's not giving back tiles either
01:10:09 <Ldp__> if I understood jburgess, he was using renderd.py as PoC and prototyping, before wrapping any changes back into the compiled renderd?
01:10:48 <IvanSanchez> dunno
01:11:19 <IvanSanchez> damn it, I just "ln -s libmapnik.so /usr/lib/libmapnik.so.0.6" and the compiled renderd seems to start
01:11:34 <IvanSanchez> problem is, if I place the lib in /usr/local/lib/ it won't work
01:11:57 <Ldp__> ldconfig ?
01:13:02 <IvanSanchez> can't tell
01:14:57 <IvanSanchez> now all I get are blank tiles, and only from localhost
01:15:21 * IvanSanchez needs some sleep in order to get over the virtualhosts stuff and mapnik xml config file tomorrow
01:37:50 <IvanSanchez> this is most impressive - everything works!!!
01:38:07 <IvanSanchez> thanks a lot for all the tips :-)
01:40:06 <Ldp__> no probs, and sleep sure sounds nice right about now
01:41:04 <IvanSanchez> so true...
01:41:52 <IvanSanchez> next week I'll try to work on this automagic mapnik stylesheet generator of mine...
01:42:04 <IvanSanchez> ... and after that, I'll *try* to set up a mapnik WMS :-)
01:42:20 <IvanSanchez> so you haven't got rid of me yet :-P
01:43:08 <IvanSanchez> good night, and thanks for all!
02:07:44 <springmeyer> IvanSanchez: cool g'night
02:07:52 <springmeyer> glad things worked for ya
02:08:09 <springmeyer> WMS is pretty easy, just RTFM :)
02:19:00 <nikq> Mapnik Trac: UbuntuInstallation edited | http://trac.mapnik.org/wiki/UbuntuInstallation?version=26
02:20:48 *** JeLuF has quit (brown.freenode.net irc.freenode.net)
02:25:33 *** JeLuF (i=jf@mormo.org) has joined #mapnik
02:27:18 <nikq> Mapnik Trac: UbuntuInstallation edited | http://trac.mapnik.org/wiki/UbuntuInstallation?version=27
02:31:15 *** aba has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** jlivni has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** nikq has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** tomhughes has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** Ldp__ has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** CIA-14 has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** w0lfie_ has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** dodobas has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** springmeyer has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** cmarqu has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** stamf has quit (brown.freenode.net irc.freenode.net)
02:31:15 *** Berteun has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** JeLuF has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** jburgess has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** tlpinney_ has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** ser has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** dukeku has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** _Dennis has quit (brown.freenode.net irc.freenode.net)
02:32:09 *** jbronn has quit (brown.freenode.net irc.freenode.net)
02:33:44 *** jburgess_ has quit (Remote closed the connection)
02:43:36 *** springmeyer (n=dane@c-24-19-50-92.hsd1.wa.comcast.net) has joined #mapnik
02:43:36 *** dukeku (i=dukeku@adhd.irule.net) has joined #mapnik
02:43:36 *** tlpinney_ (n=geo@ec2-67-202-28-190.compute-1.amazonaws.com) has joined #mapnik
02:43:36 *** ser (n=ser@sergiusz.pawlowicz.name) has joined #mapnik
02:43:36 *** jbronn (n=jbronn@70-138-113-15.lightspeed.hstntx.sbcglobal.net) has joined #mapnik
02:43:36 *** _Dennis (n=dennis@irafs1.ira.uni-karlsruhe.de) has joined #mapnik
02:43:36 *** jburgess (n=jburgess@bb-87-80-234-70.ukonline.co.uk) has joined #mapnik
02:43:36 *** JeLuF (i=jf@mormo.org) has joined #mapnik
02:43:36 *** johto_ (i=rnn@emdia.fi) has joined #mapnik
02:43:36 *** CIA-14 (n=CIA@208.69.182.149.simpli.biz) has joined #mapnik
02:43:36 *** jlivni (n=josh@76.14.74.234) has joined #mapnik
02:43:36 *** aba (i=sam@libya.turmzimmer.net) has joined #mapnik
02:43:36 *** nikq (n=nikq@li21-121.members.linode.com) has joined #mapnik
02:43:36 *** tomhughes (n=tom@gate.compton.nu) has joined #mapnik
02:43:36 *** cmarqu (i=colin@oemcomputer.oerks.de) has joined #mapnik
02:43:36 *** w0lfie_ (n=wolf@cpe-67-49-133-78.hawaii.res.rr.com) has joined #mapnik
02:43:36 *** stamf (n=stamf@116.73.233.220.exetel.com.au) has joined #mapnik
02:43:36 *** dodobas (n=dodobas@open.geof.hr) has joined #mapnik
02:43:36 *** Berteun (i=berteun@berteun.nl) has joined #mapnik
02:43:55 *** _closure has quit (Success)
02:44:04 *** johto has quit (Success)
02:44:06 *** jburgess_ (n=jburgess@92.8.252.116) has joined #mapnik
02:44:08 *** _closure_ (n=E78C8EE4@li48-120.members.linode.com) has joined #mapnik
02:44:08 <springmeyer> .startrss
02:44:08 <nikq> Okay, I'll re-start rss...
04:15:31 <w0lfie_> so
04:15:48 <w0lfie_> you know you're a geek when the first birthday greeting you get is automated from internet forums..
05:35:07 *** w0lfie_ has quit (brown.freenode.net irc.freenode.net)
05:35:07 *** CIA-14 has quit (brown.freenode.net irc.freenode.net)
05:38:26 *** CIA-14 (n=CIA@208.69.182.149.simpli.biz) has joined #mapnik
05:38:26 *** w0lfie_ (n=wolf@cpe-67-49-133-78.hawaii.res.rr.com) has joined #mapnik
05:49:45 *** w0lfie_ has quit (brown.freenode.net irc.freenode.net)
05:49:45 *** CIA-14 has quit (brown.freenode.net irc.freenode.net)
05:49:52 *** CIA-14 (n=CIA@208.69.182.149.simpli.biz) has joined #mapnik
05:49:52 *** w0lfie_ (n=wolf@cpe-67-49-133-78.hawaii.res.rr.com) has joined #mapnik
06:07:19 *** sanjiv (n=sanjiv@59.180.133.173) has joined #mapnik
06:54:19 *** johto_ is now known as johto
07:20:22 <springmeyer> happy b-day w0lfie_ :)
07:20:43 * springmeyer heads crashes
08:52:51 <w0lfie_> thanks!
10:12:51 *** jburgess_ has quit (Remote closed the connection)
10:38:23 *** Ldp__ (n=thid@osm.xs4all.nl) has joined #mapnik
11:31:39 *** sanjiv has quit (Read error: 110 (Connection timed out))
11:39:36 *** sanjiv (n=sanjiv@59.180.154.222) has joined #mapnik
11:39:40 *** sanjiv is now known as sanjiv_away
12:12:22 *** sanjiv_away is now known as sanjiv
16:07:23 * IvanSanchez loves mod_tile now that it works
17:38:36 *** sanjiv has quit (Read error: 110 (Connection timed out))
17:39:16 *** IvanSanchez has quit (Read error: 104 (Connection reset by peer))
18:23:08 <nikq> Mapnik Trac: Ticket #310 (SCons : improve output of configure stage) created | http://trac.mapnik.org/ticket/310
18:26:35 <nikq> Mapnik Trac: Ticket #311 (Add dist clean option to Scons) created | http://trac.mapnik.org/ticket/311
18:51:40 <nikq> Mapnik Trac: Ticket #303 (ogcserver: Support built in xml fallback if lxml is not available) closed | http://trac.mapnik.org/ticket/303#comment:2
19:30:08 *** IvanSanchez (n=ivan@74.3.222.87.dynamic.jazztel.es) has joined #mapnik
20:57:11 <nikq> Mapnik Trac: Changeset [1092]: slight formatting fixes to docstrings in layer class | http://trac.mapnik.org/changeset/1092
20:57:13 <nikq> Mapnik Trac: Ticket #303 (ogcserver: Support built in xml fallback if lxml is not available) closed | http://trac.mapnik.org/ticket/303#comment:2
20:57:21 <nikq> Mapnik Trac: Changeset [1092]: slight formatting fixes to docstrings in layer class | http://trac.mapnik.org/changeset/1092
21:15:35 <ser> springmeyer: ogcserver is leaking as an atlantic ocean in fcgi and wsgi as well, i've just checked it
21:16:04 * springmeyer chuckles :)
21:16:36 <springmeyer> ser: so to recap you are tracking memory usage using htop and it just keeps climbing?
21:16:45 <springmeyer> any  other observations?
21:17:36 <springmeyer> I took a quick poll of memory usage using wsgi on mac osx the other day and did not see what you describe right off
21:17:51 <ser> i think i am too stupid to understand what's going inside but if you provide me a helpful hand to switch debug on...
21:18:11 <springmeyer> I'm no expert with memory stuff but a few thoughts come to mind...
21:18:35 <ser> i can give you fast access to the server itself, i just need a ssh key, if you want of course
21:18:44 <springmeyer> 1) high memory usage != memory leak necessarily
21:19:02 <ser> but it is growing each tile is served
21:19:15 <springmeyer> sure, just have to say that :)
21:19:15 <ser> so each another tile, 1M memory more
21:19:51 <springmeyer> 2) valgrind might be a good option to look into to track what is happening at the Mapnik level
21:20:24 <ser> ok, i must learn it from the scratch, i was sure valgrind is rather for c programs
21:20:32 <ser> OK
21:20:58 <springmeyer> for python level, one sec... i had a good reference I saw
21:21:35 <ser> it should be easy as wsgi is standalone solution, easier to trace
21:22:12 <ser> but my longest python code was 5000 lines and it was awful
21:22:55 <springmeyer> yes, should be easy to trace with wsgi standalone, I agree, if you don't got off on wild goose chases :)
21:23:05 <springmeyer> okay, this looked interesting: http://www.lshift.net/blog/2008/11/14/tracing-python-memory-leaks
21:23:14 <ser> looking...
21:24:14 <springmeyer> ser: I can send you my public key, your email?
21:24:36 <springmeyer> I can't promise I'll have time soon but good to get set up in case I do
21:25:09 <ser> sergiusz@pawlowicz.name
21:25:30 <springmeyer> the first place I would test is auditing the layer_copy/style_copy methods - those would be very likely culprits
21:25:35 <springmeyer> thx
21:29:49 <ser> ok, so i am trying to run pdb
21:31:04 <springmeyer> ya, those post made some shortcut I think
21:31:11 <springmeyer> so you can't do $ pdb
21:31:13 <springmeyer> but rather
21:31:28 <springmeyer> $ python --pdb er_something :)
21:31:53 <ser> ok
21:36:46 <nikq> Mapnik Trac: Changeset [1093]: minor formatting to render_to_file docstring | http://trac.mapnik.org/changeset/1093
21:37:46 <nikq> Mapnik Trac: Changeset [1094]: + update python api docs | http://trac.mapnik.org/changeset/1094
21:39:26 <ser> through pdb it is not leaking :) :-D
21:39:48 <ser> it is a perfect solution, maybe a little slower ;-)
21:41:29 <ser> stable as a rock!
21:44:53 <ser> no, probably leaking, but slower.
21:46:15 <ser> no, ram is returning
21:46:59 <ser> it is exciting like a thriller movie!
21:47:17 <springmeyer> :)
21:50:41 <ser> ok, so i probably made a mistake, i did not restart a tilecache fcgi instance, so it took data from ogcserver fcgi, not wsgi previously. it looks wsgi is not leaking so the porblem is probably in flup VS ogcserver.
21:51:20 <springmeyer> ah interesting
21:51:43 <ser> i will confirm it switching off pdb
21:51:46 <springmeyer> ya, I've rarely used the ogcserver with fcgi
21:52:54 <ser> currently firefox is leaking ;-)
21:53:01 <springmeyer> lol
21:58:19 <ser> yes, i can confirm: no problems with wsgi, but fcgi is broken
21:58:54 <ser> no :-)
21:59:51 <ser> when i zoomed out to level 4 it took a huge amount of memory, but it returned
22:00:06 <springmeyer> cool
22:00:38 <ser> so it prevents me from other flup deployment :-)
22:00:42 <springmeyer> ya, as long as memory is not leaking and being released I figure you want huge memory usage
22:00:56 <springmeyer> right, cuz you got a ton of memory on your machines ?
22:01:02 <ser> if it is not leaking, sure!
22:01:21 <ser> i am still waiting for my 64, currently utilizing only 8
22:01:28 <springmeyer> oh okay
22:01:43 <springmeyer> ser: are you running wsgi wrapper using mod-wsgi in apache?
22:02:33 <ser> no, i am running tilecache directed to wsgi standalone now, but usually i am doing proxing through lighttpd
22:02:39 <ser> i do not use apache at all
22:03:04 <springmeyer> ah okay
22:03:14 <ser> tilecache is run as fcgi on lighttpd as well
22:03:19 <springmeyer> mod_wsgi rocks, I think it is a good reason to use apache
22:04:10 <ser> wsgi is a rather new thing, so we must wait until it will be deployed globally
22:04:28 <ser> nginx has wsgi module
22:05:08 <ser> but do did you measure mod_wsgi is really faster then wsgi standalone?
22:05:56 <ser> because in mapnik the most cpu/mem/drive is bound to postgres/tiles generation
22:06:13 <ser> so propably http connector is not a bottleneck
22:06:26 <springmeyer> I guess I'm not quite sure what you mean by wsgi standalone?
22:07:42 <ser> i mean running it through built-in http server: httpd.serve_forever()
22:08:12 <springmeyer> right, thats what I thought
22:08:48 <springmeyer> sure, I agree that performance will have more to do with the tile generation than how you serve them
22:09:02 <springmeyer> (if rendering dynamically)
22:09:42 <springmeyer> but mod_wsgi still should be much more reliable/stable/faster than the wsgiref standalone
22:10:04 <ser> yes, i saw a page describing utilizing mod_redirect for tilcache intelligent use
22:10:11 <springmeyer> because you can tune mod_wsgi daemon with the right number of processes and threads
22:10:22 <ser> i mean calling it only when there is no existing file on drive cache
22:10:40 <ser> i will deploy the same on lighttpd
22:10:59 <springmeyer> right, okay
22:11:17 <ser> springmeyer: i can also run several instances of wsgi.standalone and inteligently proxy to them
22:11:21 <springmeyer> well, I'll be stoked to see your side-by-side comparisons!
22:11:56 <ser> springmeyer: yes, i am waiting for my new server and i promise several charts
22:12:03 <springmeyer> !
22:12:29 <ser> thanks for your help!
22:12:35 <springmeyer> no problem
22:12:45 <ser> my map is staying more and more beautiful
22:12:59 <springmeyer> great
22:29:14 <nikq> Mapnik Trac: Ticket #301 (Mapnik does not issue a UserWarning when a bad (or no) datasource type is  ...) updated | http://trac.mapnik.org/ticket/301#comment:1
22:31:04 <ser> btw, i am officially registereing trademark (or patent, as you prefer) on naming multiservers: south west nort east, instead of boring a b c d :-P
22:31:34 <springmeyer> huh?
22:31:45 <ser> http://maps.pawlowicz.name/1.html
22:31:52 <ser> check URLs ;)
22:31:59 <ser> of tiles
22:32:31 <springmeyer> :)
22:34:28 <springmeyer> eww, we really need to add #305 for you
22:34:29 <nikq> Ticket #305: ogcserver: Support map buffer_size, http://trac.mapnik.org/ticket/305
22:35:47 <ser> yeah, i noticed it
22:36:03 <springmeyer> err wait you are now using TileCache to interacte with the ogcserver, so just use metatiles
22:36:44 <ser> yes, but wait a second, i will deploy WMS openlayer
22:37:30 <springmeyer> okay, them post me a patch for #305 :)
22:37:31 <nikq> Ticket #305: ogcserver: Support map buffer_size, http://trac.mapnik.org/ticket/305
22:38:22 <ser> haha
22:38:50 <nikq> Mapnik Trac: Changeset [1095]: + switch to nose as testing framework - thanks to Beau for patch and  ... | http://trac.mapnik.org/changeset/1095
22:39:28 <ser> i will use directly wms when my server arrives, my uni is horribly sloowww
22:40:15 <ser> and my department it admin is jealous of my linux skills
22:40:21 <springmeyer> ser: okay
22:40:24 <springmeyer> :)
22:40:39 <ser> so probably he will try to block me few days from his new child
22:40:41 <nikq> Mapnik Trac: Ticket #300 (Improve tests) updated | http://trac.mapnik.org/ticket/300#comment:2
22:41:06 <ser> but no worries, i'll win this game
22:42:43 <nikq> Mapnik Trac: Ticket #300 (Improve tests) updated | http://trac.mapnik.org/ticket/300#comment:3
23:03:53 <nikq> Mapnik Trac: Ticket #312 (LIB_DIR_NAME not respected for fonts install location) created | http://trac.mapnik.org/ticket/312
23:05:55 <nikq> Mapnik Trac: Ticket #180 (Parameter for line symbolizer to offset line to one side) updated | http://trac.mapnik.org/ticket/180#comment:3
23:15:34 <nikq> Mapnik Trac: Changeset [1096]: fix missing usage of LIB_DIR_NAME - closes #312 | http://trac.mapnik.org/changeset/1096
23:15:44 <nikq> Mapnik Trac: Ticket #312 (LIB_DIR_NAME not respected for fonts install location) closed | http://trac.mapnik.org/ticket/312#comment:1
23:19:37 <nikq> Mapnik Trac: Ticket #180 (Parameter for line symbolizer to offset line to one side) updated | http://trac.mapnik.org/ticket/180#comment:4
23:22:41 <nikq> Mapnik Trac: Changeset [1097]: rename python tests directory to allow for autodetection of tests when  ... | http://trac.mapnik.org/changeset/1097
23:25:13 <nikq> Mapnik Trac: Ticket #300 (Improve tests) updated | http://trac.mapnik.org/ticket/300#comment:4
23:48:56 <nikq> Mapnik Trac: Ticket #180 (Parameter for line symbolizer to offset line to one side) updated | http://trac.mapnik.org/ticket/180#comment:5
23:57:35 <nikq> Mapnik Trac: Ticket #313 (buffer parameter for PolygonSymbolizer) created | http://trac.mapnik.org/ticket/313