#mapnik log: Sunday 05, April 2009

2009 | 04

previous | next
02:23:15 *** jlivni (n=josh@76.14.74.234) has joined #mapnik
07:22:10 *** kannappans (n=chatzill@118.95.1.207) has joined #mapnik
07:39:48 *** kannappans has quit ("ChatZilla 0.9.84 [Firefox 3.0.8/2009032609]")
07:49:26 <nikq> Mapnik Trac: Changeset [1075]: wrong header names -> make dist failed | http://trac.mapnik.org/changeset/1075
10:19:03 *** cmarqu has quit ("Lost terminal")
10:58:59 <ser> hello, i cannot understand one phase of installing ogcserver
10:59:31 <ser> what is exactly "map_factory"?
10:59:54 <ser> is it just a file ?
13:01:05 <nikq> Mapnik Trac: PluginArchitecture edited | http://trac.mapnik.org/wiki/PluginArchitecture?version=14
13:28:11 <ser> ok, i can see an example on http://code.google.com/p/mapnik-utils/
14:25:20 <dodobas> yello
14:44:36 <ser> hi
14:50:49 *** sanjiv (n=chatzill@59.180.141.163) has joined #mapnik
15:21:55 <ser> wow! i have ogcserver running!
15:34:09 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik
16:18:22 *** springmeyer (n=dane@c-24-19-50-92.hsd1.wa.comcast.net) has joined #mapnik
16:20:39 *** sanjiv is now known as sanjiv_away
16:32:44 <ser> no, the speed is completely not satisfying!!
16:33:59 <springmeyer> ser: lemme guess, with the OGC Server?
16:35:08 <ser> exactly :-)
16:36:11 <ser> it is not possible to ask ogcserver for tiles, is it?
16:37:14 <springmeyer> well, it is returning 'tiles' but you must mean is it possible to request tiles by way of a uri like /tiles/x/y/z.png ?
16:38:21 <ser> ok, is possible with openlayers?
16:38:26 <ser> is it?
16:39:03 <ser> i thing i can translate it using redirect function in web server
16:39:07 <springmeyer> explain what 'it' is so I know how to answer :)
16:39:37 <ser> ok it means i currently cannot explictely ask ogcserver for /x/y/x right?
16:40:05 <springmeyer> right since it is a WMS server
16:40:20 <ser> not WMS-C? :)
16:40:38 <springmeyer> thats what TileCache does
16:40:46 <springmeyer> or one thing it can do
16:41:11 <springmeyer> ... stands in front of a WMS server and accepts TMS or Google/OSM-like tile requests
16:41:24 <ser> ok, so tilecache can translate WMS-C to WMS?
16:41:26 <springmeyer> and translates them into WMS bounding box calls
16:42:15 <springmeyer> WMS-C is caching of WMS in essence, and to do that you need to lay a predicatable grid over your map
16:42:33 <springmeyer> and thats what TileCache does, implements WMS-C
16:42:39 <springmeyer> and TMS
16:42:50 <ser> ok, now i can understand, thanks :)
16:43:06 <ser> so i am doing fast deployment of tilecache
16:43:35 <springmeyer> what do you mean?
16:44:42 <ser> i hope it would make things faster as rendering smaller tiles should be faster? and of course plus cache od drive or memcache?
16:45:03 <ser> currently i have http://maps.pawlowicz.name/osm.html?zoom=15.100185528681108&lat=6704174.00999&lon=-26681.21598&layers=B
16:45:11 <springmeyer> also, be careful to 'judge' speed in light of the zoom level
16:45:15 * springmeyer looks
16:45:48 <ser> and in bigger zoom it is not bad, but when i am trying to show the whole UK, it takes ages
16:46:29 <springmeyer> right, the UK is a large area - asking Mapnik to run the __really__ complex filters on that large an area takes time
16:47:12 <springmeyer> and any tiles with the high resolution coastline includes in the bbox are going to be much slower than
16:47:13 <ser> and everything is disk-related, IO times are unacceptable
16:47:24 <springmeyer> even queries that include larger areas but no coastlines
16:47:35 <ser> springmeyer: but i hope they would be cached
16:48:06 <springmeyer> ser: what, the tiles or something else?
16:48:13 <ser> tiles in such a zoom
16:48:19 <springmeyer> my sense from you earlier is you want not to cache things
16:48:57 <ser> previously i have an idea not to cache, but currently it looks it would be just technically impossible
16:49:00 <springmeyer> (but of course, up to you :) as you learn )
16:49:13 <springmeyer> right.
16:50:13 <springmeyer> it is technically possible to serve data very fast for large areas, but OSM data is both large, varied, and has complex cartographic filters being applied
16:50:40 <springmeyer> but anyway, I'm very interested in seeing good profiling done
16:51:01 <springmeyer> of where time is being take up on rendering at different zoom levels
16:51:15 <ser> springmeyer: i will try to do my best after i learn it enough of course
16:51:31 <springmeyer> as part of your testing and research sure would be cool to see some benchmarks written down :)
16:51:36 <springmeyer> you bet
16:51:37 <ser> is it possible to ask tilecache to cache only from a prticular zoom?
16:52:04 <springmeyer> good question, I'm not sure if there is a switch for tilecache like that out of the box
16:52:14 <springmeyer> but if could be added fairly easiely
16:52:18 <ser> ok, i will investigate it
16:52:54 <springmeyer> using TileCache in front of a WMS layer has gotten more attention than the tilecache direct->to->mapnik layer
16:53:23 <springmeyer> ser: also, how did you deploy the OGC server?
16:53:39 <ser> as fcgi
16:53:57 <ser> i am running concurrent 16 processes
16:54:12 <ser> but defintely the problem is in drive I/O
16:54:28 <springmeyer> ser: remove...
16:54:29 <springmeyer> map.fractionalZoom = true;
16:54:39 <springmeyer> and singleTile:true,
16:54:50 <springmeyer> then 'tiles' will be requested through WMS
16:55:22 <ser> ok, i did not learn openlayes enough yet, thanks for a direction
16:57:32 <ser> yes, it works, but as you said: it is not faster
16:58:53 <ser> here you can see an overview of load and disk i/o: http://pawlowicz.name/MyServers
16:59:07 <ser> this server is house
16:59:20 <springmeyer> this will help a tiny bit: http://openlayers.org/dev/examples/multiserver.html
17:00:25 *** sanjiv_away is now known as sanjiv
17:00:56 *** aub has quit ()
17:01:25 <ser> but currently it is load average: 30.33, 17.22, 8.95 so you can see pretty nothing :(
17:02:08 <ser> yes, i will deploy a multiserver in few minutes, thanks
17:02:30 <ser> i've already have dns entries for it
17:02:32 <springmeyer> ser: have you created indecies on your osm tables and run VACUUM ANALYZE ?
17:02:49 <ser> i did not do VACUUM ANALYZE
17:03:25 <ser> can i run it during import of minutely mapnik or i should rather switch it off temporary?
17:04:12 <springmeyer> beyond my experiences - use your judgement and let me know how it goes :)
17:04:33 <ser> OK :)
17:05:27 <springmeyer> fyi: http://dpaste.com/24433/
17:05:57 <ser> thank you!
17:06:15 <ser> i am copying it to my wiki asap :)
17:06:46 <springmeyer> thats just python btw, so you do >>> settings = {'dbprefix':'osm_planet'} or whatever
17:07:40 <ser> sure, do not worry, i have basic knowledge :)
17:08:16 <springmeyer> gocha, cool
17:08:16 <ser> i would not afraid to ask if i cannot understand something ;-)
17:08:41 <springmeyer> good! you'll know more than I before you know it then :)
17:09:07 <ser> haha
17:09:32 <ser> curently as i can see indexes are about 50% of my postgres
17:09:43 <springmeyer> wow
17:09:47 <springmeyer> ser: also note that rcoup recently added some optimization options to the postgis plugin
17:10:13 <springmeyer> but AFAIK nobody has played with/documented them in relation to OSM data
17:10:22 <ser> you mean it is not in 0.6.0?
17:10:37 <springmeyer> if you are intereste you might post to the mailing list with questions if he is not around
17:10:51 <springmeyer> nope, they were included as options in 0.6.0
17:11:08 <ser> ok, so i will look inside, thanks
17:12:00 <springmeyer> ya.
17:12:02 <ser> thanks for all, chap, it really speeds up my learning process :)
17:12:13 <springmeyer> I tried to document them lightly but I have not used yet
17:12:31 <springmeyer> ( ie cursor_size, and row_limit)
17:12:31 <springmeyer> http://dpaste.com/24440/
17:13:13 * springmeyer sees the defaults I wrote likely make no sense ;(
17:13:36 <springmeyer> ugh
17:13:37 <ser> i cannot understand it right now, but i just need learn more about it
17:30:58 <ser> ok, multiserver works from a to z :-D
17:32:15 <ser> but the server itself does not ;-)
17:33:54 <ser> before i started to play with OSM i was sure i have a new, fast and shiny server
17:51:11 <ser> btw, to monitor server in realtime, i am using:
17:51:13 <ser> alias statz='dstat -D sda -M load,disk,net,cpu,mem,page,swap,time'
18:02:20 *** sanjiv_ (n=chatzill@59.180.147.96) has joined #mapnik
18:02:50 *** sanjiv has quit (Read error: 110 (Connection timed out))
18:03:32 *** sanjiv_ is now known as sanjiv
18:47:41 <nikq> Mapnik Trac: Ticket #291 (make build fails due to -lagg not being found) updated | http://trac.mapnik.org/ticket/291#comment:2
18:57:16 *** sanjiv has quit ("ChatZilla 0.9.84 [Firefox 3.0.8/2009032711]")
18:59:00 <nikq> Mapnik Trac: Ticket #291 (make build fails due to -lagg not being found) updated | http://trac.mapnik.org/ticket/291#comment:3
19:07:28 <nikq> Mapnik Trac: Ticket #293 (Add Kismet to Scons build and wrap in python bindings (with docstring)) created | http://trac.mapnik.org/ticket/293
19:25:23 <nikq> Mapnik Trac: Ticket #291 (make build fails due to -lagg not being found) updated | http://trac.mapnik.org/ticket/291#comment:4
19:43:28 <nikq> Mapnik Trac: Changeset [1076]: fixes Ticket #291 - make build fails due to -lagg not being found (I  ... | http://trac.mapnik.org/changeset/1076
20:47:19 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik
21:02:12 <nikq> Mapnik Trac: Changeset [1077]: forget to escape path | http://trac.mapnik.org/changeset/1077
21:23:16 *** rcoup (n=rcoup@ip-118-90-84-72.xdsl.xnet.co.nz) has joined #mapnik
22:12:54 <nikq> Mapnik Trac: Ticket #291 (make build fails due to -lagg not being found) updated | http://trac.mapnik.org/ticket/291#comment:5
22:18:40 <springmeyer> rcoup: hey have you ever chatted with dominic about your deb packaging?
22:19:04 <rcoup> erm, nope.
22:19:15 <rcoup> whenever i have to majorly redo them i just start with his
22:19:20 <springmeyer> as you know its all fairly new to me how it works, but I know that he is working on packaging 0.6.0 this weekend
22:19:26 <rcoup> ah ok
22:19:27 <springmeyer> k
22:19:41 <springmeyer> maybe you guys efforts overlap - just a thought :)
22:20:09 <springmeyer> he let me know that we missed the cutoff I for jaunty
22:20:10 <rcoup> There's a time & a place for hurting your head with packaging
22:20:16 <rcoup> never & nowhere :P
22:20:22 <springmeyer> :)
22:20:51 <springmeyer> well, I took a look at all of his patches (numerous) against Scons for 0.5.1 for debian
22:20:52 <rcoup> easiest way to waste a day is waiting on pbuilder & friends to build stuff across multiple distros & architectures
22:21:03 <springmeyer> and made switches for em
22:21:18 <springmeyer> ewww- sounds not up my alley
22:23:03 <springmeyer> anyway, just wanted to let you know he's working on things
22:23:43 * springmeyer wants to make sure we time releases in the future as informed (where possible) by debian schedules
22:26:03 <rcoup> springmeyer: https://wiki.ubuntu.com/KarmicReleaseSchedule
22:27:30 * aba wants to remind that Ubuntu is a fork from Debian, and didn't take over Debian
22:29:18 <rcoup> aba: i wasn't suggesting otherwise, just that Ubuntu has a defined schedule with hard dates every 6 months, and Debian (AFAIK) doesn't.
22:29:48 <aba> rcoup: well, Debian releases when the software is bug-free enough ...
22:29:53 * aba shuts up now
22:30:59 <rcoup> :)
22:31:57 <springmeyer> rcoup: brilliant - good on ya :)
22:32:38 <rcoup> springmeyer: so if i understand correctly, your release has to be in for FeatureFreeze, and you can do micro-releases up to BetaFreeze
22:32:58 <rcoup> and past that if stuff is really broken.
22:33:11 <springmeyer> rcoup: thanks I was about to repeat that exact thing to make sure
22:33:22 <rcoup> but /me is not an expert
22:51:49 *** springmeyer has quit ()
22:55:17 <nikq> Mapnik Trac: Ticket #291 (make build fails due to -lagg not being found) updated | http://trac.mapnik.org/ticket/291#comment:6
22:57:26 <ser> rcoup: you can use lenny/backports
22:58:26 <ser> although i do not know any procedures how to put something there
23:00:26 <aba> ser: www.backports.org has the procedures
23:00:42 <aba> basically: versions that are in testing can be put there (but please see the web page for details)
23:56:21 <ser> imho it is worth to think about it as another debian stable will be in one year ;-)