02:51:54 *** Alireza_Kashian has quit (Read error: 110 (Connection timed out)) 03:15:59 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik 04:34:04 *** aub has quit () 05:29:18 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik 06:32:29 *** aub has quit () 09:50:37 *** Alireza_Kashian (n=chatzill@91.98.183.62) has joined #mapnik 10:15:29 *** Alireza_Kashian has quit ("ChatZilla 0.9.84 [Firefox 3.0.5/2008120122]") 13:32:28 *** D3f0 has quit (Remote closed the connection) 14:01:52 *** D3f0 (n=defo@190.176.209.54) has joined #mapnik 14:51:16 *** kunitoki (n=kraken@host210-49-dynamic.7-79-r.retail.telecomitalia.it) has joined #mapnik 15:19:47 *** kunitoki has quit ("leaving") 15:43:59 *** D3f0 has quit (Read error: 104 (Connection reset by peer)) 16:20:36 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik 16:37:13 *** aub has quit () 17:09:41 *** D3f0 (n=defo@190.176.196.145) has joined #mapnik 17:19:46 *** kunitoki (n=kraken@host210-49-dynamic.7-79-r.retail.telecomitalia.it) has joined #mapnik 18:07:50 <nikq> Mapnik Trac: Ticket #219 (python bindings getting installed to incorrect location) closed | http://trac.mapnik.org/ticket/219#comment:6 18:13:50 <nikq> Mapnik Trac: Ticket #115 (Update Install Document on Mapnik.org by release and trunk) updated | http://trac.mapnik.org/ticket/115#comment:6 18:14:21 <nikq> Mapnik Trac: Ticket #115 (Update Install Document on Mapnik.org by release and trunk) updated | http://trac.mapnik.org/ticket/115#comment:7 18:17:13 <nikq> Mapnik Trac: Ticket #217 (Add ability to set PKG_CONFIG_PATH in SCons) updated | http://trac.mapnik.org/ticket/217#comment:8 18:25:11 <nikq> Mapnik Trac: Ticket #70 (Ability to set label value manually in TextSymbolizer (rather than have ...) updated | http://trac.mapnik.org/ticket/70#comment:6 18:39:29 *** rcoup (n=rcoup@ip-118-90-96-141.xdsl.xnet.co.nz) has joined #mapnik 19:03:44 *** artem (n=artem@i-83-67-142-225.freedom2surf.net) has joined #mapnik 19:17:00 *** artem has quit () 20:07:01 *** aub (n=aubrey@cpe-72-227-134-148.nyc.res.rr.com) has joined #mapnik 20:37:40 *** artem (n=artem@i-83-67-142-225.freedom2surf.net) has joined #mapnik 20:45:21 <kunitoki> artem: ping 20:45:33 <artem> hey kunitoki 20:45:38 <kunitoki> hey man :) howdy 20:45:46 <artem> not bad :) 20:45:56 <kunitoki> cool ! 20:46:10 <kunitoki> i've got some code here you may want to look at and consider 20:46:34 <kunitoki> you know the ideas about sqlite bloba and mapnik geometries ? 20:46:40 <artem> sure 20:46:59 <kunitoki> well, apart the use of the index... i've got a running input plugin 20:47:28 <kunitoki> you may want to throw some code for optimizing and make use of the r*tree 20:47:35 <artem> great, lets add index 20:47:56 <kunitoki> yeah 20:48:00 <kunitoki> this is how i do 20:48:15 <kunitoki> in the blob i write geometries using this schema 20:49:18 <kunitoki> http://pastebin.com/d686c9e48 20:49:52 <kunitoki> probably for making good use of the index, data have to be organized differently. 20:49:55 <kunitoki> i don't know actually 20:50:35 <kunitoki> i make use also of an external table (geocatalog) for defining extents of the tables, which gets queryed to fill the envelope of the table 20:51:14 <kunitoki> that one can be updated using a trigger, probably 20:51:40 <artem> kunitoki: looks good 20:52:13 <kunitoki> wait i throw in a patch so you can have a look 20:53:44 <jburgess> the pastebin looks al awful lot like wkb. Is there any advantage over using wkb instead? 20:57:21 <artem> jburgess: yes, good point. might be better to stick to WKB as a default. But I like idea to be able to use custom blobs. Perhaps, using integers for storing coord-s. 20:58:22 <kunitoki> yeah i thought about wkb, but that kind of stuff is plain linear and reading from it and pushing a geometry2d is pretty straightforward 20:58:29 <kunitoki> with minimal conversion 20:59:08 <jburgess> ok, but other than the definition of your header, the storage of the co-ordinates with the number of points _is_ wkb 20:59:12 <kunitoki> and that kind of stuff works for single and multi, points line and polygons, all treated in the same way, no switches 20:59:21 <kunitoki> ok 20:59:46 <jburgess> take a look at the wkb reader inside mapnik to see how it interprets the data it gets from postgis 20:59:57 <kunitoki> i actually haven't looked how wkb is done internally 21:01:29 <kunitoki> well yes 21:02:26 <kunitoki> ok that is probably better i think 21:03:46 <artem> kunitoki: using wkb will make creating r*tree index easier - no need for custom functions to extract bbox from geoms 21:03:53 <artem> for sqlite 21:03:55 <kunitoki> ok usre 21:04:30 <kunitoki> what i need now is being able to dump a shape file into a sqlite database 21:04:40 <kunitoki> so to do more massive testing 21:04:54 <artem> use spatialite utility : .loadshp 21:05:10 <kunitoki> ok 21:06:22 <artem> to create r*tree index : create virtual table spatial_idx using rtree (id,minx.maxx,miny,maxy); 21:07:29 <artem> then you can populate index table with : insert into table_spatial_idx select id, MbrMinX(geom),.... 21:09:33 <kunitoki> ok 21:20:43 <Mrfo> whats the benefit of using sqlite with mapnik over something like postgres? 21:20:56 <kunitoki> embeddability ? 21:20:57 <artem> sqlite is just a file 21:21:09 <artem> server-less SQL database 21:21:12 <Mrfo> kunitoki, ah yeah thats a good point 21:21:31 <kunitoki> the main problem i face with oracle is that i can't package my server in a usbkey and deploy an application somwhere lese 21:21:41 <kunitoki> :D 21:21:59 <Mrfo> that would be pretty sweet for a distributed system 21:23:11 <kunitoki> yeah, if you think sqlite supports transactions... what a shapefile can't 21:26:55 <kunitoki> artem: what i do not understand actually is: the r*tree is present in sqlite or only in spatialite ? 21:27:27 <artem> rtree is a part of sqlite3 21:27:51 <kunitoki> ok 21:28:19 <kunitoki> how can i know the extent of that table ? 21:28:44 <artem> it is quite neat - you can use saptialite to create and fiddle with geometries but for reading sqlite3 is good enough 21:29:03 <kunitoki> i used an external table where i keep: create table geocatalog (table_name text, minx real, miny real, maxx real, maxy real) 21:29:27 <kunitoki> so i can have the extent of the table even if i don't have a function that calculates that 21:29:41 <kunitoki> but is precalculated 21:29:52 <artem> kunitoki: I'm not sure there's a way at the moment but better check. 21:30:45 <artem> kunitoki: i might be an idea to allow explicit layer extent, same as in postgis 21:31:18 <artem> kunitoki: extent calculations are always slow unless cached. 21:31:54 <kunitoki> ok 21:32:06 <kunitoki> but what if one doesn't specify it 21:32:41 <artem> well , in this case we can calculate it 21:33:20 <artem> but it'll require iterating through all records 21:34:02 <kunitoki> yes, no way 21:34:13 <kunitoki> i'll keep both approaches 21:34:18 <artem> ok 21:34:34 <kunitoki> if you specify a geometry_catalog, then it will use that support table in the sqlite database 21:34:44 <kunitoki> if you specfy the extent... it will use that 21:34:51 <artem> sounds good to me 21:59:53 *** artem has quit () 22:00:33 <nikq> Mapnik Trac: Ticket #223 (Add sqlite3.input plugin for loading geometries from blob columns) created | http://trac.mapnik.org/ticket/223 22:09:27 *** artem (n=artem@i-83-67-142-225.freedom2surf.net) has joined #mapnik 22:17:41 <kunitoki> mmmh 22:18:06 <kunitoki> now i'm using geometry_utils::from_wkb 22:18:25 <kunitoki> but there is only empty geometries 22:22:28 <jburgess> can you dump out the hex of the data? maybe the first 32 bytes or so. 22:22:50 <jburgess> I think the wkb reader is rather silent about data it doesn't like 22:23:05 <kunitoki> ok 22:23:15 <kunitoki> any way to dump a feature ? 22:23:34 <kunitoki> in a easy way... i can see to_string() but that's only for properties 22:23:41 <jburgess> dunno about sqlite, in postgres you can just select the geometry and it'll give you the it in hex 22:23:49 <kunitoki> ah ok 22:24:04 <kunitoki> there is the AsString() function 22:24:12 <kunitoki> or AsText 22:24:18 <kunitoki> or you want the binary ? 22:24:19 <jburgess> that is ok, so long as it doesn't jkust give you the WKT 22:24:19 <Mrfo> what about AsBinary() 22:24:51 <jburgess> It would be worth checking the WKT makes sense first, but yes, the binary is what the wkb reader should be parsing 22:25:09 <kunitoki> 22:25:25 <kunitoki> well... it can't be printed 22:25:29 <kunitoki> obviously :) 22:25:39 <jburgess> is there no way to turn the bytes into hex? 22:26:17 <jburgess> even if it takes a bit of printf("%02x) 22:26:18 <Mrfo> theres an encode function in postgis 22:26:46 <Mrfo> and you can give it a base there 22:26:51 <kunitoki> MULTIPOLYGON(((1069931.104586 562646.99585 then thousands of ordinates 22:26:59 <jburgess> maybe just sticking some debug code into the geometry_utils::from_wkb to print all it receives as hex 22:27:00 <Mrfo> but i dont know if that helps with sqlite 22:27:44 <kunitoki> or a geometry_utils::to_wkb ? 22:28:24 <jburgess> it would be worth checking if the to_wkb gets that text string or the binary encoded data it is expecting 22:29:02 <jburgess> maybe I should update my SVN tree and try myself. Is any of this code committed? 22:29:10 <artem> kunitoki: you can use hex(geom) in sqlite 22:30:18 <kunitoki> artem: thanx you always come and save stuff :) 22:30:37 <kunitoki> probably is only a bad map extent 22:31:17 <artem> I had a problem with extent calculation on 2.2 . 2.3 seems to work 22:31:31 <kunitoki> here i'm using 2.3 22:31:37 <artem> ok 22:32:05 <kunitoki> jburgess: wait i'll doa patch shortly 22:33:18 <kunitoki> 0001FFFFFFFF7AAF5D3EA32E394100F314D14B76E0C0DBFAA356DE30394100000000000010007C02000000020000007AAF5D3EA32E3941804FA06BD22BE0C0DBFAA356DE30394100F314D14B76E0C0FE 22:33:53 <kunitoki> here it is a hex(geom) of the first road of roads.shp loaded as table in spatialite 22:34:17 <kunitoki> 0001 should be linestring... probably ? 22:34:50 <kunitoki> ah no 22:36:24 <jburgess> first byte should be the endianness 22:36:37 <jburgess> then 4 bytes for the geometry type 22:38:50 <jburgess> Take a look at a file named 'ZMSgeoms.txt', it should be part of your postgis install 22:39:05 <kunitoki> well... here i have no postgres :) 22:39:58 <jburgess> that gives a breakdown of the wkb structure, ok I'll pastebin a relevant bit 22:40:03 <kunitoki> http://trac.mapnik.org/ticket/223 22:42:05 <jburgess> kunitoki: http://pastebin.ca/1331124 22:42:44 <jburgess> the bit about embedding the SRID in the WKB is a postgis extension 22:43:12 <kunitoki> mmmh 22:43:34 <kunitoki> so probably spatialite wkb isn't compatible with geometry_utils::from_wkb 22:44:08 <jburgess> it does not look like it, but may need to look closer at sqlite to see what it is doing 22:45:50 <kunitoki> http://www.gaia-gis.it/spatialite/spatialite-2.2_manual.html#t3.2 22:49:43 <jburgess> that certainly looks like WKB... section 3.3 is probably relevant 22:50:01 <kunitoki> yes it seems also to me 22:50:08 <kunitoki> i wonder what happens then 22:50:13 <artem> http://pastebin.ca/1331130 22:50:48 <artem> hmm ?? 22:51:30 <kunitoki> whuho ! 22:51:44 <jburgess> artem: look about half way down the sqlite string and you'll see the start of the WKB "0200000006000000..." 22:51:50 <kunitoki> there is some parts equals... 22:52:27 <jburgess> the sqlite seems to add the SRID and BBOX at the front, 22:52:33 <artem> looks like sqlite3 has its own header :) 22:53:18 <kunitoki> 39bytes header... but then we miss the first endian 22:53:59 <jburgess> It would probably be fairly easy to add a new method into the geometry::to_wkb which takes the sqlite format 22:54:20 <jburgess> just read the endian, skip rest of header then parse rest as wkb 22:56:08 <kunitoki> yes 22:56:33 <kunitoki> i can do: wkb_reader(const char* wkb,unsigned size,unsigned offset_start) 22:56:42 <kunitoki> wkb_reader(const char* wkb,unsigned size,unsigned offset_start=0) 22:57:57 <kunitoki> or a from_sqlitewkb ? 22:58:05 <jburgess> that would make more sense 22:58:15 <kunitoki> mmmh ok 22:58:26 <jburgess> it may be good to grab the srid and bbox too, maybe adding them as new properties in the geometry 22:58:47 <jburgess> I know that postgis can also return a slight more enhanced binary format which includes the SRID 23:00:46 <jburgess> I think it may also have one with a bbox too. I saw a mention of that in the postgis code 23:02:45 <kunitoki> yes that could be good to have too 23:03:03 <kunitoki> i think about a last parameter of from_wkb 23:03:34 <kunitoki> which takes a enum wkb_format { Postgis, Sqlite, OGC, Generic, Unknown }; 23:05:08 <jburgess> could do. I'm not sure which would be better: generic API + enum, vs different method call for each type 23:05:45 <jburgess> I guess a generic API is probably the cleaner way to do it 23:05:53 <kunitoki> that is a matter of taste, but since the function gets too many parameters already... 23:06:18 <kunitoki> yeah, a parameter is always a good way to go in these cases 23:06:35 <kunitoki> but having more than 6 or 7 arguments is baaaaad 23:09:17 <jburgess> In this case I think all the inputs will be "const char *" + length. 23:09:55 <jburgess> artem: I was wondering about geometry_utils::from_wkb(). Why is multiple_geometries there, why not enable it all the time? 23:11:41 <artem> it affects how text label placements are calculated - at least it used to be . 23:13:55 <jburgess> ok, I'll try enabling it in the OSM postgis datasources and see if it has any effect 23:14:24 <jburgess> I don't see why it should be a problem. For the non-multigeom case, the reader behaves exactly the same 23:14:44 <kunitoki> it doesn't work the same 23:15:11 <jburgess> no, where is the difference? The only differences seemed to be for the multi-geom handling 23:15:12 <kunitoki> it seems the wkb_reader is too lazy about what we feed to it 23:15:41 <kunitoki> multi_geometry will add multiple geometry2d to the feature 23:15:48 <kunitoki> and they all get a label 23:16:11 <kunitoki> without multiple_geometry the feature have only one geometry2d expressed in multiple move_to commands 23:16:26 <jburgess> can you point out where this is? 23:16:28 <kunitoki> and it gets only one label 23:16:58 <kunitoki> you can see the difference read_multipoint and read_multipoint_2 23:17:19 <kunitoki> it is used in the datasource parameters everywhere 23:17:29 <kunitoki> (the multiple_geometries) 23:18:59 <jburgess> ok, I think I was reading the code incorrectly before 23:19:56 <jburgess> I thought what happened was that if were reading a multigeometry without the flag set then we'd ignore everything after the first instance (i.e. throwing away all the 'multi' aspect of it) 23:20:15 <kunitoki> ah no 23:20:59 <kunitoki> i previously encountered both cases while i wrote the ogr plugin. it was clear how they got represent in the map when seeing to the labels 23:24:34 <kunitoki> anyway please give my patch a revision, things are there but it needs some patching elsewhere in wkb_reader. i'm going to bed now 23:25:00 <kunitoki> i really want to have this usable soon ! 23:25:06 <artem> kunitoki: great, thanks I'll have a look tomorrow! 23:25:11 <jburgess> The code makes more sense now. We either read it as lots of geometries or as a single combined one with gaps. 23:25:28 <artem> jburgess: yes 23:26:22 *** rcoup_ (n=rcoup@ip-118-90-27-63.xdsl.xnet.co.nz) has joined #mapnik 23:26:25 <kunitoki> byez 23:26:28 *** kunitoki has quit ("Lost terminal") 23:30:26 *** artem has quit () 23:33:44 *** kunitoki (n=kraken@host210-49-dynamic.7-79-r.retail.telecomitalia.it) has joined #mapnik 23:33:51 <kunitoki> wh 23:33:56 <kunitoki> eh 23:34:13 <kunitoki> look at the image 23:34:24 <kunitoki> http://trac.mapnik.org/attachment/ticket/223/wrong_demo.png 23:34:24 *** rcoup has quit (Read error: 110 (Connection timed out)) 23:34:35 <kunitoki> it is produced only out of sqlite 23:34:41 <kunitoki> it is clearly wrong 23:35:03 <kunitoki> but something is visible at least 23:35:36 <kunitoki> it is clearly a problem in wkb_reader 23:35:53 <kunitoki> it needs a refinement... i'll do it properly tomorrow 23:36:36 <jburgess> looks pretty good to me! 23:36:41 <kunitoki> if you can attach to ticket 223 some more info on how to patch my code in order to make use of the r*tree would be perfect ! 23:36:41 <nikq> Ticket #223: Add sqlite3.input plugin for loading geometries from blob columns, http://trac.mapnik.org/ticket/223 23:37:04 <kunitoki> jburgess: it is wrong anyway 23:37:43 <kunitoki> well probably not 23:37:53 <kunitoki> i changed the extent 23:38:16 <kunitoki> it should look like this (http://trac.mapnik.org/raw-attachment/ticket/170/demo.png) 23:39:18 <jburgess> looking again at the sqlite format page, the multigeoms are different 23:39:30 <jburgess> aka "collection entity" 23:39:55 <kunitoki> wkbGeometryCollection isn't handled in mapnik 23:40:02 <kunitoki> anyway 23:40:13 <kunitoki> case wkbGeometryCollection: break; 23:40:18 <jburgess> I think your issue may be due to not setting the extent? 23:40:27 <jburgess> your seeing a zoomed out view of the data 23:40:59 <kunitoki> yes ! it works ! 23:41:24 <jburgess> great 23:44:16 <nikq> Mapnik Trac: wkb-sqlite.patch attached to Ticket #223 | http://trac.mapnik.org/attachment/ticket/223/wkb-sqlite.patch 23:44:52 <kunitoki> apply both the patches 23:45:04 <kunitoki> anyway i will revise the wkb tomorrow 23:46:35 <kunitoki> night ! 23:47:00 *** kunitoki has quit ("Lost terminal")