#mapnik log: Wednesday 04, February 2009

2009 | 02

previous | next
01:53:03 *** jbaird has quit ()
02:38:35 *** rcoup has quit ()
03:02:22 *** jbaird (n=jbaird@c-24-23-238-70.hsd1.ca.comcast.net) has joined #mapnik
03:17:05 *** jbaird has quit ()
03:23:42 *** jbaird (n=jbaird@c-24-23-238-70.hsd1.ca.comcast.net) has joined #mapnik
03:27:51 *** rcoup (n=rcoup@ip-118-90-78-242.xdsl.xnet.co.nz) has joined #mapnik
03:51:09 *** racicot has quit ("ChatZilla 0.9.84 [Firefox 2.0.0.19/2008121623]")
03:57:52 *** forrestv has parted #mapnik ("Leaving")
04:10:36 *** jbaird has quit ()
05:22:28 *** racicot (n=chatzill@dsl-209-166-85-189.whidbey.net) has joined #mapnik
05:23:29 *** jbaird (n=jbaird@c-24-23-238-70.hsd1.ca.comcast.net) has joined #mapnik
06:12:47 *** D3f0 has quit ("Konversation terminated!")
07:40:34 *** rick68_ has quit ()
07:58:53 *** xcacou (n=aga@vel78-2-82-243-106-36.fbx.proxad.net) has joined #mapnik
08:05:56 *** jbaird has quit ()
08:22:11 *** kunitoki (n=55121f9a@67.159.35.76) has joined #mapnik
09:19:19 *** Jon___ (n=Jon@host81-130-15-214.in-addr.btopenworld.com) has joined #mapnik
09:26:12 <kunitoki> morning.... got a question
09:26:27 <kunitoki> how i'm supposed to use the singleton<> class ?
09:27:47 <kunitoki> cause i've defined it like this http://pastebin.com/d41e59d27
09:27:57 <kunitoki> but my constructor doesn't get called
09:28:24 <kunitoki> i can use my own (much more easier) version but i want to stick to mapnik code style and classes
09:31:11 <kunitoki> any ideas ? i looked at the datasource_cache class which is a singleton, but i don't see any interesting bits
09:32:51 <kunitoki> maybe like this ?
09:33:29 <kunitoki> http://pastebin.com/d5df4572f
09:36:21 <kunitoki> yesh the second one is better- but i'm supposed to correctly releasing the object in the singleton destructor ?
09:47:45 <kunitoki> no, the destructor is never called
09:54:46 <kunitoki> and should be called atexit from what i see, but it doesn't
10:25:35 *** D3f0 (n=defo@190.176.193.117) has joined #mapnik
10:27:10 *** rcoup has quit ()
10:34:22 <kunitoki> anyone ?
10:56:31 <nikq> Mapnik Trac: occi-input-plugin-3.patch attached to Ticket #212 | http://trac.mapnik.org/attachment/ticket/212/occi-input-plugin-3.patch
11:00:06 <nikq> Mapnik Trac: Ticket #212 (Add OCCI.input plugin to read all ORACLE (> 10g) supported vector formats) updated | http://trac.mapnik.org/ticket/212#comment:2
11:25:04 <kunitoki> artem_: look here http://www.gaia-gis.it/spatialite/
11:30:08 <kunitoki> artem_: maybe we can borrow some code and implement a spatialite input plugin ?
11:42:08 <kunitoki> springmeyer: forward the links to artem when he is onle
11:42:11 <kunitoki> online
11:43:05 <kunitoki> i will start to write an input plugin for that
12:03:03 *** kunitoki has quit ("CGI:IRC")
13:21:29 *** kunitoki (n=4f0731d2@67.159.35.76) has joined #mapnik
13:38:55 *** D3f0 has quit (Remote closed the connection)
14:15:21 *** D3f0 (n=defo@190.176.222.197) has joined #mapnik
15:04:03 *** artem (n=artem@i-83-67-142-225.freedom2surf.net) has joined #mapnik
15:16:15 <artem> kunitoki: ping
16:02:50 *** xcacou has quit (Remote closed the connection)
16:13:47 *** D3f0 has quit (Read error: 113 (No route to host))
17:00:11 <kunitoki> artem: pong
17:01:35 <kunitoki> i started implementing spatialite as input plugin ... http://www.gaia-gis.it/spatialite-2.3/
17:02:01 <kunitoki> i really want to have something like this, cause i want a transition to a more embeddable database
17:05:52 <artem> kunitoki: good, I was planning to work on slite3 plug-in next week :)
17:06:21 <artem> may be we can combine efforts
17:08:20 <scruggs> a sqlite3 plugin would be great!
17:08:43 <artem> also, it would be better to read directly from sqlite3 there is no need for spatialite really
17:11:07 <scruggs> I've been successful using mapnik to render directly from shapefiles on my omap3 board but I suspect sqlite would be more efficient
17:12:49 <artem> yes, it should
17:13:31 <crschmidt> Depending on how the geometry is stored... it may be more efficient to read geometry data from shapefiles than reading WKT or similar from sqlite3 and converting to a geometry internally...
17:13:43 <artem> I'm trying to evaluate if storing large data sets works with sqlite3+R*Tree
17:14:15 <kunitoki> have you seen the spatialite stuff ?
17:14:20 <artem> I think it should works well for < 1e6 records
17:14:33 <artem> kunitoki: yes, it appears quite buggy
17:14:40 <kunitoki> yes ?
17:14:49 <artem> at least on os x
17:15:01 <artem> this can be due to GEOS
17:15:28 <kunitoki> mmmh yeah, here is working ok with the tests
17:15:52 <kunitoki> i didn't tried anything more than that anyway
17:16:00 <artem> I was trying with OSM data and it borks quite a lot
17:16:00 <kunitoki> have you tried 2.3 ?
17:16:11 *** springmeyer has quit ()
17:16:17 <artem> hmm ..  checking
17:16:19 <kunitoki> http://www.gaia-gis.it/spatialite-2.3/
17:16:39 <kunitoki> it is alpha but they have fixed lot of stuff in the meanwhile
17:16:55 <artem> ok, I'm using 2.2 , I'll updato 2.3, cheers:)
17:17:57 <artem> still, for just reading geometries stored as blobs and indexed using rtree there is no need for spatialite at all - it's just an overhead
17:19:35 <artem> The requested URL /spatialite-2.3/test-2.3.sqlite.gz was not found on this server.
17:20:00 <artem> kunitoki: did you manage to get 2.3 ?
17:22:36 <kunitoki> well no i only tested 2.2 so far
17:22:54 <kunitoki> i found 2.3 five minutes ago
17:22:56 <artem> crschmidt: geometries are stored as blobs, there is also VirtualShape ext which supports connecting to raw shapefiles through sqlite3 and using sqlite3 indexing caps
17:23:22 <artem> links are dead :(
17:25:00 <kunitoki> artme: sure it ia a overhead, but it implements good features for a spatial database
17:25:27 <kunitoki> like spatial predicates (contains, intersect, anyinteract and others)
17:25:50 <kunitoki> i know only for storing geometries would be ok.
17:26:37 <kunitoki> i can use a blob also in oracle and save a lot of overhead, but then i miss oprators like relate, validate, lrs and the like
17:27:38 *** racicot has quit ("ChatZilla 0.9.84 [Firefox 2.0.0.19/2008121623]")
17:28:39 <artem> kunitoki: but you can still use spatialite - the data is just plain sqlite3 file
17:29:14 <artem> and geometries are stored as WKB (blobs) or WKT (text)
17:33:35 <kunitoki> ok, so what are your plans ?
17:33:54 <kunitoki> for a sqlite spatial implementation ?
17:35:25 <artem> kunitoki: it would be really cool to have a framework for different "blob" readers. And support both sqlite3 and spatialite (which is OGC SQL)
17:38:39 <kunitoki> yeah sure, also would be an idea to use the boost::serialization library in order to serialize binary objects directly, so we can store mapnik::geometry2d directly
17:38:55 <kunitoki> in a sqlite3 blob
17:40:07 <kunitoki> artem: while you think, here it is http://trac.mapnik.org/ticket/212#comment:2
17:40:16 <kunitoki> the last improvements
17:40:21 <kunitoki> on OCCI :)
17:40:38 <artem> fantasic!
17:40:47 <artem> fantastic:)
17:41:22 <kunitoki> next week i put togheter some more information on how to build that, and put in the wiki
17:41:38 <kunitoki> and also i'll put a png of the result i've been able to do so far
17:41:47 <kunitoki> with Venice data :)
17:42:37 <artem> kunitoki: great!
17:47:17 <kunitoki> i have only one thing to ask you
17:47:20 <kunitoki> about my patch
17:47:35 <artem> yes>
17:47:36 <artem> ?
17:47:52 *** jbaird (n=jbaird@shiva.mochimedia.net) has joined #mapnik
17:47:57 <kunitoki> singleton class in utils.hpp
17:48:17 <kunitoki> can you look at my occi_environment class ?
17:48:44 <kunitoki> i don't think it will be ever relased (the destructor is not called, in debug mode i don't see the clog output)
17:49:48 *** Jon___ has quit ()
17:50:00 <kunitoki> http://trac.mapnik.org/attachment/ticket/212/occi-input-plugin-3.patch#L82
17:50:54 <kunitoki> i don't understand why it is not called neither the constructor nor the destructor
17:51:11 <kunitoki> and so i cannot call something atexit
17:51:29 <kunitoki> (which is release the Environment)
17:54:37 <artem> hmm.. i think atext is called after program exited so no debug out ??
17:54:59 <kunitoki> ah ok
17:55:13 <artem> i would check, though
17:55:14 <kunitoki> but it is ok to write the class like that ?
17:55:27 <artem> ok, let me have a look
17:58:53 <artem> kunitoki: looks fine, I'll have a look properly later on. One thing I noticed in ogr.input is that OGRFieldDefn* fld = def->GetFieldDefn (i); is called for every record. I guess these calls are just returning cached object ?
17:59:21 <artem> sorry. wrong method ..
17:59:42 <artem> whis one :) OGRFeatureDefn* def = layer_.GetLayerDefn();
18:00:40 * artem bbl
18:05:55 *** rcoup (n=rcoup@ip-118-90-96-141.xdsl.xnet.co.nz) has joined #mapnik
18:08:11 *** kunitoki has quit (K-lined)
20:03:04 *** jbronn_ (n=jbronn@70-138-113-15.lightspeed.hstntx.sbcglobal.net) has joined #mapnik
20:12:48 *** rcoup has quit (Remote closed the connection)
20:13:25 *** rcoup (n=rcoup@ip-118-90-96-141.xdsl.xnet.co.nz) has joined #mapnik
20:14:35 *** jbronn has quit (Read error: 110 (Connection timed out))
20:41:03 *** Jon___ (n=Jon@93-97-6-206.zone5.bethere.co.uk) has joined #mapnik
20:48:22 *** kunitoki (n=kraken@host210-49-dynamic.7-79-r.retail.telecomitalia.it) has joined #mapnik
20:50:50 <Mrfo> is there a way to specify the geometry column via xml?
20:56:13 <artem> Mrfo: for postgis input you should be able to  <Parameter name="geometry_filed">the_geom</Parameter>
20:56:26 <Mrfo> awesome
20:56:54 <Mrfo> how recently was that added in?
20:57:15 <artem> let me check..
20:57:51 <Mrfo> i ask because im using the windows binaries for this project
20:58:02 <Mrfo> and i feel like thats a recent feature
21:00:28 <artem> Mrfo: r769
21:00:29 <nikq> http://trac.mapnik.org/changeset/769, at , by artem: + applied patch from rcoup to allow specify geometry column.
21:00:35 <Mrfo> ahh
21:00:56 <artem> Mrfo: looks like you have to wait till 0.6.0 win32
21:01:18 <rcoup> Mrfo: and it's "field" not "filed", but you probly guessed that ;)
21:01:30 <artem> rcoup: sure :)
21:01:43 <Mrfo> i see. thank guys
21:02:45 <artem> rcoup: I'm yet to apply your last postgis patch, just didn't have a time. Would it work with latest trunk ?
21:02:49 *** JuergenL has quit ("Verlassend")
21:03:17 <rcoup> artem: don't see why not. if you have any problems, just comment on the ticket and i'll update the patch
21:03:27 <artem> cool, thanks
21:37:15 *** jbaird has parted #mapnik ()
21:39:19 *** racicot (n=chatzill@dsl-209-166-85-189.whidbey.net) has joined #mapnik
22:49:07 <artem> kuntuki: storing mapnik geometry directly as an option sounds great btw
22:49:38 <artem> kunitoki
22:58:58 <kunitoki> artem: pong
22:59:08 <artem> ping
22:59:18 <kunitoki> yeah, i'm doing some test with tha
22:59:21 <kunitoki> t
23:00:42 <kunitoki> have you got chance to look at boost serialization library ?
23:01:03 <artem> it might be even better to write custom to/from bin logic, geometry in mapnik just a sequence of (x,y,command)
23:01:26 <kunitoki> yeah sure that is even better
23:02:26 <kunitoki> we can think a way so we cn write a layer on top of that, and let the input plugin (datasource) decide how to treat it
23:02:42 <artem> yep
23:02:42 <kunitoki> i think that we can do it once for sqlite, postgres, mysql, oracle
23:03:22 <kunitoki> so one can store its object in any database, taken into account the db have to offer proper indexing of the geometry
23:04:40 <artem> sounds good
23:05:04 <kunitoki> how you would implement a proper extent based query on that blob ?
23:05:16 <kunitoki> you have to unify also the index access ?
23:06:21 <artem> two-stage query - first rtree based then internally in mapnik ??
23:06:47 <artem> shape file reader does something similar
23:06:55 <kunitoki> yeah
23:07:19 <artem> or, custom func in sqlite3
23:07:47 <kunitoki> yeah i've seen the docs and examples around adding special function to sqlite3
23:08:03 <artem> in fact r*tree index store geom extent already for us
23:08:06 <kunitoki> that is pretty clean and slick way of doing things
23:08:39 <kunitoki> what about postgis ?
23:09:18 <kunitoki> you will use a special field on index that ?
23:09:27 <kunitoki> on=and
23:11:06 <artem> postgis might need some help:) I think it needs some methods to extract extent from the blob.
23:11:22 <artem> a blob
23:11:43 <kunitoki> ok
23:12:31 *** Jon___ has quit ()
23:16:07 <kunitoki> i think it can also be transparentto the input plugin abi
23:16:57 <kunitoki> the datasource / featureset interface shouldn't be aware of that, but load things from a blob field only if it is specified in the datasource parameters
23:17:51 *** rcoup has quit (Read error: 110 (Connection timed out))
23:18:12 <kunitoki> the only problem could be interface a query to the index, and you have to carefully choos how to do it, cause some underlying database implementation could not have what you want
23:18:59 *** jbronn_ is now known as jbronn
23:21:25 <kunitoki> we could put in a blob the binary content of a single geometry as we read from a shape file, so we can borrow the reading / transformation code from that
23:21:43 <kunitoki> just to make some tests
23:24:36 <kunitoki> ok time to sleep
23:25:28 <kunitoki> we will speak on that tomorrow if i get time
23:25:32 <kunitoki> nito
23:25:35 *** kunitoki has quit ("Lost terminal")
23:37:07 *** springmeyer (n=springme@75-165-113-214.tukw.qwest.net) has joined #mapnik
23:41:31 *** artem has quit ()