Setup

Installing the library

Geomancer can be installed using pip.

$ pip install geomancer

This will install all dependencies for every data warehouse we support. If you wish to do this for only a specific warehouse, then you can add an identifier.

$ pip install geomancer[bq] # For BigQuery
$ pip install geomancer[sqlite] # For SQLite
$ pip install geomancer[psql] # For PostgreSQL (*In Progress*)

Alternatively, you can also clone the repository then run install.

$ git clone https://github.com/thinkingmachines/geomancer.git
$ cd geomancer
$ python setup.py install

Setting-up your data warehouse

We highly-recommend using BigQuery as your data warehouse and Geofabrik’s OSM catalog as your source of Points and Lines of interest.

  1. First, download a `.shp.zip` file for your Region-of-Interest (ROI). In this example we can choose the Philippines (.shp.zip)

Note

You can definitely choose multiple ROIs as long as you keep them in separate contexts. A BigQuery best practice is to put them in separate datasets. So if you will use Geomancer in US and in Japan, you should have two datasets in your project (e.g., us_osm and jp_osm).

  1. Then, convert all your shape files into WKT because BigQuery accepts WKT files. For example, let’s convert gis_osm_pois_free_1.shp into WKT. You can use ogr2ogr for this kind of task:
$ ogr2ogr -f CSV gis_osm_pois_free_1.csv gis_osm_pois_free_1.shp -lco GEOMETRY=AS_WKT

You can do this for all shapefiles you currently have. At the end of this operation, you should have a CSV file (with a WKT) column for all files.

  1. Load your files into BigQuery. Our best practice is to just use the CSV filename as the table name. Refer to the bq load instructions in the Google Cloud Platform documentation. In this example, we’ll load all WKT files in a dataset called ph_osm
$ bq load --source_format=CSV    \
          --skip_leading_rows=1  \
          --autodetect           \
          ph_osm.gis_osm_pois_free_1 gis_osm_pois_free_1.csv

Geomancer assumes that your polygons are of the type STRING. Thus, we recommend passing the --autodetect option when loading to BigQuery.

And that’s it! When casting a spell, you can then use bigquery://project-name for your dburl and project-name.ph_osm.gis_osm_pois_free_1 as your source_table.

Using other data warehouses aside from BigQuery

You can use other data warehouses. This enables us to support SpatiaLite and PostGIS data warehouses. In order for them to interface with Geomancer, it should have the following characteristics:

  • The column where geometries are stored should be of type STRING and named WKT
  • It should support GIS functions (ST_GeomFromText, ST_Distance, ST_Length, etc.)

Using other source datasets aside from OSM

Do you have other geospatial data? Want to use Geomancer on them? It is possible, as long as they fulfill the following requirements:

  • They should have a column of geometries
  • There must be a unique identifier or a primary key for each row
  • There should be a way of filtering them properly

In this way, it is then possible to query from other datasets. For example, the DistanceToNearest spell accepts an argument on. You can add a colon : to specify which column this spell will filter upon (default is fclass). For example:

>>> DistanceToNearest("embassy", **kwargs)   # Will filter embassy in fclass (default)
>>> DistanceToNearest("user_group:4G-users") # Will filter 4G-users in user_group