04:50:23 *** mapniklog (~mapniklog@li21-121.members.linode.com) has joined #mapnik
04:50:23 <hubbard.freenode.net> Topic for #mapnik is: Mapnik - C++/Python Toolkit | Mapnik 0.7.0 released
04:50:23 <hubbard.freenode.net> Users on #mapnik: mapniklog HuskyRunner cgs_bob gavinf ser tomhughes CIA-29 mishok13 StormTide hobu racicot dodobas cmarqu crust jburgess mapnikbuild jbronn nikq dukeku
05:06:21 *** Phurl (~mdupont@ip-81-210-228-126.unitymediagroup.de) has joined #mapnik
05:09:44 *** HuskyRunner has quit (Quit: Leaving.)
06:15:43 *** HounD (~HounD@unics1.grfc.ru) has joined #mapnik
06:17:16 *** HounD has quit (Client Quit)
06:18:51 *** HounD (~HounD@unics1.grfc.ru) has joined #mapnik
06:24:52 *** jbronn has quit (Read error: Operation timed out)
06:28:29 *** jbronn (~jbronn@70-138-113-15.lightspeed.hstntx.sbcglobal.net) has joined #mapnik
06:29:02 *** HounD has parted #mapnik (None)
06:49:39 <dodobas> yello
07:02:58 *** gavinf has quit (Ping timeout: 248 seconds)
07:43:06 *** Ldp__ (~thid@osm.xs4all.nl) has joined #mapnik
12:22:52 *** HounD (~HounD@unics1.grfc.ru) has joined #mapnik
12:23:02 *** HounD has parted #mapnik (None)
13:40:29 *** HuskyRunner (~HuskyRunn@66-219-8-179.ip.gvtel.com) has joined #mapnik
13:51:45 *** luneff (~yury@87.117.62.0) has joined #mapnik
13:52:47 *** bitterman (~yury@87.117.62.0) has joined #mapnik
13:54:14 <bitterman> Ldp__, thanks for yesterdays help to me as luneff :) finally it worked
13:54:44 *** bitterman has quit (Client Quit)
13:55:11 *** bitterman (~yury@87.117.62.0) has joined #mapnik
13:56:08 *** luneff has quit (Disconnected by services)
13:56:14 *** bitterman is now known as luneff
13:57:31 <Ldp__> luneff: so, what was it?
13:58:06 <luneff> foolish mistake as it always is. i need to write a rule of thumb that says "if it doesn't work, swap lon and lat"
13:58:19 <luneff> bounding box was not right )
13:58:30 *** HuskyRunner has parted #mapnik (None)
13:58:34 <Ldp__> ah! :)
13:58:46 <luneff> and only eagles eye could see that from the code i pasted )
13:58:50 <luneff> but it was possible )
13:58:56 *** HuskyRunner (~HuskyRunn@66-219-8-179.ip.gvtel.com) has joined #mapnik
13:59:00 <luneff> the bb was somewhere in Iran, i guess
13:59:05 <luneff> hard to tell by shore line
13:59:38 <luneff> and the map differs slightly from the one i see on tah.openstreetmap.org ) but it's ok
14:00:15 <luneff> i don't like pythons crashing during rendering, but maybe later i'll put it everything on linux and it should do ok )
14:00:32 <Ldp__> tah is an entirely different map and renderer
14:00:48 <luneff> i've thought it was done by mapnik )
14:00:55 <Ldp__> so I wonder what "slightly" means to you :)
14:01:00 <Ldp__> tah = osmarender
14:01:32 <luneff> hm. tah is better, i guess. icons for buildings are larger and seem from less zoom
14:01:50 <luneff> some streets are bold while in mapnik they are less )
14:01:55 <luneff> but i can live with that )
14:02:14 <Ldp__> different approaches, and you can make a mapnik-based map as crowded as the tah map
14:02:23 <Ldp__> it's just that we keep the osm.xml clean
14:02:32 <luneff> i see :)
14:03:25 <luneff> so for "cons" to production use of mapnik for me i have crashing python and some random psql errors during rendering
14:03:40 <luneff> and for "pro" that i managed to render my map )
14:03:52 <luneff> 23k files for 100 square kilometers
14:04:35 <Ldp__> still wonder why you get sql errors
14:05:01 <luneff> anyways, i like the feature that i can choose what i need to render and in which zoom so i can render full russia map with less zoom and selected cities with more zoom. that's cool )
14:05:30 <luneff> they are just random happening, i can give full text if i see one, if you'd like
14:08:33 <luneff> Ldp__, http://pastebin.com/3bAYUw69
14:09:45 <luneff> and after that python crashed
14:10:35 <Ldp__> that's not helpful. It doesn't point out where and why the error occured
14:10:47 <luneff> python 2.5.4. maybe i need to upgrade it, i just couldn't find more recent msi python version
14:11:20 <luneff> that's only thing i can give ) in java world i could do more with log4j )
14:13:24 <Ldp__> can you run that query by hand, in psql?
14:13:38 <luneff> just a moment
14:14:54 <luneff> column "
14:14:55 <luneff> highway" does not exist
14:15:05 <luneff> that's default.style problem, i guess
14:15:12 <luneff> but it is as recent as it could be
14:15:30 <Ldp__> \d planet_osm_line
14:15:34 <Ldp__> you should have lots of columns
14:16:35 <luneff> there is "highway text"
14:17:10 <luneff> ah..
14:17:17 <luneff> that was a copy&paste problem
14:17:23 <luneff> splitted "" column
14:17:29 <luneff> it works, just returns nothing
14:18:05 <Ldp__> then you have no data in that tile
14:18:09 <Ldp__> s/tile/bbox/
14:18:24 <Ldp__> but that shouldn't crash python
14:18:27 <luneff> ok, but why is that a psql error?
14:18:36 <luneff> i guess that are synchronization problems
14:18:36 <Ldp__> don't know
14:18:43 <luneff> so i'll try newer python for windows
14:18:54 <luneff> from activestate as python.org doesn't have one
14:18:58 <Ldp__> maybe you should try to keep to 1 thread
14:19:36 <luneff> how do i do that?
14:20:19 <Ldp__> generate_tiles.py
14:20:19 <Ldp__> # Default number of rendering threads to spawn, should be roughly equal to number of CPU cores available
14:20:20 <Ldp__> NUM_THREADS = 4
14:20:28 <Ldp__> set it to 1
14:22:36 <luneff> no errors as i can see
14:23:07 <luneff> so new python version might help? perhaps, they fixed some threading code
14:23:24 <Ldp__> prehaps
14:23:26 <Ldp__> perhaps
14:23:41 <Ldp__> or it is an issue on windows
14:23:59 <luneff> maybe. win7 x64 is a bit extreme, i guess )
14:24:21 <luneff> of all the software i have, only one is unusable here )
14:24:28 <luneff> ireport for jasperreports )
14:24:33 <luneff> so i use xp mode for that
14:36:47 <luneff> it still crashes. so, i guess, one thread, one governor, ein computer )
14:40:30 <luneff> i've got a question. how much space does large rendered map take? for ex, a map of russia from 1 to 14 zoom?
14:43:12 <Ldp__> 1 to 14, not that much, relatively
14:44:18 <luneff> like what? 1g? 10g?
14:44:28 <luneff> or just 512m?
14:45:57 <Ldp__> at a guess, less than 1GB
14:46:06 <luneff> ok, thank you )
14:46:29 <Ldp__> but I'm not sure. Every additional zoom quadruples your tiles, and on average your storage needs
14:46:45 <Ldp__> Although higher zooms have less objects, so will compress better, overall
14:47:07 <Ldp__> and Russia is kind of big :)
14:47:28 <luneff> yes, it surely is )
14:47:44 <luneff> maybe, i even don't need all russia in this project
14:48:09 <luneff> just to make "between cities with installations" not empty squares )
14:50:38 *** Phurl is now known as Phurlv4
14:55:05 *** HuskyRunner has quit (Quit: Leaving.)
14:56:09 <luneff> thanks again for the help :)
14:56:19 <luneff> you've made more than one guy happy :)
15:40:12 *** luneff has quit (Quit: Leaving)
15:46:05 *** shoe (~user@ip24-250-63-6.ri.ri.cox.net) has joined #mapnik
15:53:01 *** gavinf (~gavinf@196.211.2.133) has joined #mapnik
16:01:52 *** filbertkm (~chatzilla@184.sub-75-227-60.myvzw.com) has joined #mapnik
16:02:21 *** luneff (~yury@87.117.62.0) has joined #mapnik
16:21:36 *** springmeyer (~springmey@c-76-28-156-154.hsd1.wa.comcast.net) has joined #mapnik
16:26:42 *** HuskyRunner (~HuskyRunn@66-219-8-179.ip.gvtel.com) has joined #mapnik
16:27:01 <StormTide> mornin
16:28:37 <HuskyRunner> hello StormTide
16:46:42 *** gavinf has quit (Ping timeout: 265 seconds)
16:51:42 *** HuskyRunner has quit (Quit: Leaving.)
16:52:22 *** dkb45 (~dkb45@66-219-8-179.ip.gvtel.com) has joined #mapnik
16:55:29 *** dkb451 (~dkb45@66-219-8-179.ip.gvtel.com) has joined #mapnik
16:58:22 *** dkb45 has quit (Ping timeout: 256 seconds)
17:03:00 *** jctull (~jctull@adsl-75-14-218-16.dsl.renocs.sbcglobal.net) has joined #mapnik
17:07:16 <StormTide> springmeyer, are other calculations than the scale_denominator dependant on the pixel-size-in-meters constant of .00028? I'm considering changing it to a more accurate number for mobile display. (the iPhone screen is 160or so ppi, whereas that constant works out to around 90 ppi)
17:08:54 *** peter_ (~peter@moobilenet-109-97.ucdavis.edu) has joined #mapnik
17:09:53 <peter_> im having toruble rendering postgis tables with quantumnik. anyone able to assist?
17:10:37 <springmeyer> peter_: yes
17:11:21 <springmeyer> peter_: you getting errors about SRID mismatch?
17:11:30 <peter_> thanks! i have no problem with shapefiles.
17:11:39 <peter_> no errors just a blank render
17:12:37 <springmeyer> oh, okay
17:13:00 <peter_> i was suspicious about the srid  so i did a  shp2pgsql with the same shapefile that works.
17:13:11 <peter_> with no luck
17:13:21 <springmeyer> okay, thats odd
17:13:25 <springmeyer> what Mapnik version?
17:15:41 <peter_>  0.6.1 used the install instructions from here http://trac.mapnik.org/wiki/UbuntuInstallation
17:16:06 <peter_> using Karmic
17:16:43 <springmeyer> okay, so you installed from packages....
17:17:20 <springmeyer> peter_: in Mapnik 0.7.0 we added better error reporting when problems happen in the postgis plugin
17:17:38 <springmeyer> so with 0.6.1 is it harder to know what is wrong
17:17:51 <springmeyer> without looking into the postgres logs
17:18:33 <peter_> ok any good instructions for removing packaged mapnik and installing the latest stable version?
17:18:50 <springmeyer> what was the exact command line you used to import the data with shp2pgsql ?
17:18:57 <StormTide> you can remove it with apt-get remove pretty easy
17:19:12 <StormTide> dpkg -l |grep -i mapnik should get you the right package name
17:19:19 *** cgs_bob has quit (Ping timeout: 276 seconds)
17:19:21 <springmeyer> sure if you want to re-install just do apt-get remove libmapnik0.6 mapnik-utils python-mapnik
17:20:12 <peter_> ./shp2pgsql -s 26910 ~/Documents/SWIM/srwp_bdat srwp_bdat > ~/Documents/SWIM/bdat.sql
17:20:32 <Ldp__> shp2pgsql -s <projection code> -g way <shapefile base> <table name> | psql -U <user> -q <database>
17:24:44 <springmeyer> okay, so peter_ what is your QGIS preference say for...
17:24:51 <springmeyer> preferences > crs > ?
17:27:44 <peter_> this is from edit > QGIS Options > CRS +proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs
17:31:00 <springmeyer> change it to "prompt" for projection....
17:31:20 <springmeyer> so it should ask you when the projection is not able to be known automatically
17:31:38 <springmeyer> avoiding the default to the ^^ above projection, which is the problem
17:32:08 <springmeyer> so that down the road you would get errors about SRID mismatch from postgis...
17:35:56 *** dkb451 has parted #mapnik (None)
17:38:11 *** dkb (~dkb@66-219-8-179.ip.gvtel.com) has joined #mapnik
17:39:35 *** mperry (~mperry@c-67-164-175-65.hsd1.co.comcast.net) has joined #mapnik
17:41:25 *** filbertkm has quit (Ping timeout: 276 seconds)
17:42:23 <peter_> ok, so changed to prompt, added postgis layer (which did not prompt the crs prompt-- it appears that the srid is read by QGIS). the same issue arises )
17:44:24 <peter_> layer properties in QGIS show the correct srid for the postgis data
17:44:53 *** tcarobruce (~tcarobruc@adsl-75-10-247-30.dsl.pltn13.sbcglobal.net) has joined #mapnik
17:50:12 <springmeyer> peter_: okay, now...
17:50:33 <dodobas> yello
17:50:34 <springmeyer> can you http://dpaste.com the output of Plugins > Quantumnik > View Live XML
17:50:46 <springmeyer> heya dodobas
17:52:39 <peter_> http://dpaste.com/164244/
17:53:34 *** cgs_bob (~bob@132.sub-75-208-90.myvzw.com) has joined #mapnik
17:54:38 <dodobas> springmeyer: good morning :)
17:55:06 <springmeyer> good afternoon :)
17:55:50 <dodobas> we shoud switch time zones for a week or two.. :)
17:56:07 <springmeyer> ya, keep things interesting :)
17:56:15 <springmeyer> peter_: that looks good, hmmm
17:59:41 <springmeyer> peter_: at this point the next things I would try are either:
17:59:50 <springmeyer> rebuild with Mapnik 0.7
17:59:59 <springmeyer> or send me the shapefile so I can test with 0.7
18:02:35 <peter_> ok another weird thing is that changing the default crs to prompt causes the shapefile version to be blank when rendered with mapnik renderer
18:04:15 <peter_> once i changed it back to global +proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs shapefile works in the renderer
18:05:05 <peter_> i'll do both how should i get you the shapefile?
18:05:25 <springmeyer> dbsgeo@gmail.com
18:05:40 * springmeyer is on a call now, will take a look sometime today
18:05:46 <springmeyer> zip it
18:06:02 <peter_> ok thanls very much!
18:08:55 <dodobas> springmeyer has clients, do not disturb him... :)
18:36:28 *** dkb has parted #mapnik (None)
18:42:16 *** gavinf (~gavinf@196.211.119.210) has joined #mapnik
19:14:05 *** gavinf has quit (Quit: gavinf)
19:14:37 *** gavinf (~gavinf@196.211.119.210) has joined #mapnik
19:20:41 *** gavinf has quit (Quit: gavinf)
19:21:51 *** Ldp__ has quit (Ping timeout: 245 seconds)
19:25:33 *** Ldp__ (~thid@osm.xs4all.nl) has joined #mapnik
19:47:20 *** dkb (~dkb@66-219-8-179.ip.gvtel.com) has joined #mapnik
20:09:52 *** racicot has quit (Quit: ChatZilla 0.9.86 [Firefox 2.0.0.21pre/2009020912])
20:15:35 <springmeyer> peter_: did you send an email?
20:15:51 *** shoe has quit (Remote host closed the connection)
20:19:05 *** jctull has quit (Quit: jctull)
20:48:00 <peter_> just did thanks again....
20:59:41 *** shoe (~user@ip24-250-63-6.ri.ri.cox.net) has joined #mapnik
21:01:32 <springmeyer> okay, I'll take a look tonight or in the morning, off now
21:01:39 *** springmeyer has quit (Remote host closed the connection)
21:01:55 *** jctull (~jctull@adsl-75-14-218-16.dsl.renocs.sbcglobal.net) has joined #mapnik
22:14:56 <dkb> would someone have suggestions on where to look for performance bottlenecks when testing mod_tile with OpenLayers?  To me it almost looks like the server is waiting until all the tiles are rendered before attempting to send any?
22:19:33 <jburgess> dkb: the tiles get rendered in blocks of 8x8 so it probably does look like you see nothing then lots appear at once
22:20:02 <jburgess> rendering them like this is several times more efficient then rendering the tiles individually
22:21:57 <dkb> the server just seems slow, I'm even getting broken links occasionally because the tiles are rendered in time. I have ModTileRequestTimeout 15
22:22:00 <jburgess> the time to render each tile depends on both the amount of data in the area and the complexity of the rendering style. Some zoom levels are slower than others
22:22:22 <dkb> yes, I noticed the zoom/complexity relation
22:23:07 <jburgess> whole much data do you have loaded in the DB?
22:23:15 <dkb> just the state of virginia
22:23:29 <dkb> or bbox around that state
22:23:35 <jburgess> if you run "vmstat 1" you should get an idea about whether it is bottlenecked on the CPU or disk
22:24:55 <jburgess> If only one state then you can probably get it all loaded into RAM in a few hundred MB so it sounds like the CPU is probably a bottleneck for you
22:25:47 <dkb> http://pastebin.com/raw.php?i=3i8QBXVG
22:25:52 <jburgess> what are the hardware specs like?
22:27:27 <jburgess> I'm guessing from that output it is a dual-core or hyperthreaded single core with a few GB of ram?
22:27:41 <dkb> Xeon 3060 (the same as a Core2 Duo E6600 with different name)
22:27:59 <dkb> 3GB RAM
22:28:21 <jburgess> ok, my home machine has a E6600 w/ 4GB
22:28:57 <jburgess> If you move the map around a lot without waiting for the tiles to appear then the CPU should get up to 100%
22:30:38 <jburgess> Under normal conditions the majority of requested tiles have already been rendered before. This is especially true of the lower zoom tiles which tend to be slower to generate (zoom 0 - 12 ish)
22:31:18 <jburgess> Starting with a completely empty cache is close to the worst case
22:33:05 <jburgess> Some people like to pre-render all the tiles for a range of zooms in the typical bounding box to avoid this
22:33:32 <jburgess> generally once you have rendered a bunch of tiles you keep them aroud for a long time
22:33:57 <Ldp__> unless you're one of the stubborn people that like to expire by deleting :)
22:35:06 <dkb> ok, so I would call use generate_tiles script to generate zoom level range already.. the cache IS basically empty now
22:36:37 <dkb> the CPU does go close to 100%
22:37:12 <dkb> I remember seeing some thread #defines in code/config somewhere and wondered if that needed to be adjusted
22:39:30 <jburgess> the is a NUM_THREADS define but the default of 4 is still OK on a dual core machine
22:42:17 <jburgess> generate_tiles is not the one to use
22:42:42 <jburgess> since it does not create the .meta tiles but only plain PNGs
22:45:15 <dkb> ok, one of the bins/scripts in mod_tile source directory I assume
22:45:46 <jburgess> yes, though nothing is quite designed to easily fulfill this role
22:45:56 <jburgess> the closest is probably speedtest.cpp
22:46:10 <jburgess> this will render all tiles over a range of zoom levels for a given bounding box
22:46:30 <jburgess> the flaw is that the bbox and zoom range is hard coded in the source
22:47:26 <jburgess> alternative the render_all utility will render all tiles with a given zoom and x/y tile co-ordinates from the command line, but it won't accept something nice like a bounding box in degrees
22:47:28 <dkb> ok, I will watch for that and recompile for correct values
22:51:32 *** jctull has quit (*.net *.split)
22:51:32 *** dkb has quit (*.net *.split)
22:53:17 *** jctull (~jctull@adsl-75-14-218-16.dsl.renocs.sbcglobal.net) has joined #mapnik
22:53:17 *** dkb (~dkb@66-219-8-179.ip.gvtel.com) has joined #mapnik
22:55:39 *** ajashton (~aj@c-69-136-229-112.hsd1.dc.comcast.net) has joined #mapnik
23:06:14 <Ldp__> o hai jburgess, did you read my PM about the coastlines?
23:06:53 <jburgess> Ldp__: yes I saw it, does your new file fix the broken bits around the US?
23:07:01 <Ldp__> I haven't deployed them myself :(
23:07:12 <Ldp__> gimme a moment
23:10:51 <dkb> Do those broken bits have anything to do with the square blotches off the south east coast of the US?
23:11:03 <Ldp__> them's the ones :)
23:11:40 <dkb> how did they get there to begin with?
23:12:16 *** jctull has quit (Quit: jctull)
23:13:31 <Ldp__> either a real broken coastline, or a broken tool
23:14:18 <dkb> reimporting with the updated  shp file to postgresql will be enough to fix it?
23:14:32 <Ldp__> these shapes don't make it into postgis
23:15:28 <Ldp__> deploying is as simple as putting the new shapefile in the right spot, and clearing the cache
23:16:35 <Ldp__> jburgess: http://tile.openstreetmap.nl/?zoom=4&lat=32.74393&lon=-45.14721&layers=B000000
23:16:56 <Ldp__> no large issues, but I see a few small blocks of water inland
23:17:07 <Ldp__> Mexico, Albania
23:17:16 <Ldp__> Greece has a large block
23:18:52 <Ldp__> probably genuine coastline errors
23:20:29 <peter_> can anyone tell me if legending is implemented in 0.7.0
23:20:57 <jburgess> Ldp__: it is looking a lot better
23:21:45 <jburgess> Ldp__: are those shapefiles are the URL you posted to me?
23:21:50 <Ldp__> yes
23:22:01 <Ldp__> http://mijndev.openstreetmap.nl/~ldp/coastlines/
23:22:42 <jburgess> the timestamps still show as this morning, do they have the later diff from the 24th?
23:23:23 <Ldp__> I'm currently generating coastlines from planet-100224, but it's still in the highzoom bits, so will take a few more hours
23:23:40 <jburgess> ok, just wanted to check they match what we just saw on the map
23:23:46 <Ldp__> timestamps are from when I actually tarballed them
23:24:08 <jburgess> thanks, I'll grab them now.
23:24:22 <Ldp__> that's why I'm generating that 'generated_from' file as well
23:54:51 <jburgess> Ldp__: I have pulled in the shpaefiles and forcing the render of z0-z9. It looks better now, e.g. http://www.openstreetmap.org/?lat=34.8&lon=-71.5&zoom=4&layers=B000FTF
23:55:35 <Ldp__> yes, lots
23:57:37 <Ldp__> have you seen http://svn.mapnik.org/branches/0.7.1-dev/ ?
23:57:38 *** mperry has quit (Read error: Connection reset by peer)
23:59:14 <jburgess> I have not tried it yet. I was planning to look at it one weekend when I have some time to ensure everything works fine.
23:59:28 <jburgess> I tried 0.7 out at home and didn't see any issues