Cores

The Core module contains various engines that can interact with different databases. We strive for Geomaner to be database-agnostic: a query should ideally be executable across all types of data warehouse.

geomancer.backend.cores.base

Base class for all DBCore implementations

A DBCore is simply a database backend. It can be BigQuery, PostGIS, or an SQLite database. Whenever you want to add a new DBCore, simply subclass from the base DBCore class and implement the required methods

class geomancer.backend.cores.base.DBCore(dburl, options=None)[source]

Bases: abc.ABC

Base class for all DBCore implementations

ST_GeoFromText(x)[source]

Custom-implementation of converting a string into a geographical type

As it turns out, ST_GeogFromText only exists in BigQuery and PostGIS. Only ST_GeomFromText is available for Spatialite. Thus, we need to construct our own method for type-casting

__init__(dburl, options=None)[source]

Initialize the database core

Parameters:
  • dburl (str) – Database url used to configure backend connection
  • options (geomancer.backend.settings.Config, optional) – Specify configuration for interacting with the database backend. Auto-detected if not set.
get_engine()[source]

Get the engine from the DBCore

Returns:Engine with the database dialect
Return type:sqlalchemy.engine.base.Engine
get_tables(source_uri, target, engine)[source]

Create tables given a sqlalchemy.engine.base.Engine

Parameters:
  • source_uri (str) – Source table URI to run queries against.
  • target (pandas.DataFrame or str) – Target table to add features to. If a string, must point to a table location found in the database.
  • engine (sqlalchemy.engine.base.Engine) – Engine with the database dialect
Returns:

Source and Target table

Return type:

(sqlalchemy.schema.Table, sqlalchemy.schema.Table)

load(df)[source]

Load a pandas.Dataframe into the database

Parameters:df (pandas.DataFrame) – Input dataframe
Raises:NotImplementedError – This is an abstract method

geomancer.backend.cores.bq

class geomancer.backend.cores.bq.BigQueryCore(dburl, options=None)[source]

Bases: geomancer.backend.cores.base.DBCore

BigQuery DBCore

client

google.cloud.client.Client – BigQuery client for handling BQ interactions

ST_GeoFromText(x)[source]

Custom-implementation of converting a string into a geographical type

As it turns out, ST_GeogFromText only exists in BigQuery and PostGIS. Only ST_GeomFromText is available for Spatialite. Thus, we need to construct our own method for type-casting

__init__(dburl, options=None)[source]

Initialize the database core

Parameters:
  • dburl (str) – Database url used to configure backend connection
  • options (geomancer.backend.settings.Config, optional) – Specify configuration for interacting with the database backend. Auto-detected if not set.
load(df, dataset_id, expiry=3, max_retries=10, **kwargs)[source]

Upload a pandas.DataFrame as a BigQuery table with a unique 32-char ID

Parameters:
  • df (pandas.DataFrame) – Input dataframe to upload to BigQuery
  • dataset_id (str) – ID to name the created Dataset
  • expiry (int, None) – Number of hours for a given table to expire. Default is 3.
  • max_retries (int) – Number of retries for the upload job to ensure that the table exists. Default is 10
Returns:

The full path for the created table

Return type:

str

geomancer.backend.cores.sqlite

class geomancer.backend.cores.sqlite.SQLiteCore(dburl, options=None)[source]

Bases: geomancer.backend.cores.base.DBCore

SQLite Core with Spatialite Extension

ST_GeoFromText(x)[source]

Custom-implementation of converting a string into a geographical type

As it turns out, ST_GeogFromText only exists in BigQuery and PostGIS. Only ST_GeomFromText is available for Spatialite. Thus, we need to construct our own method for type-casting

__init__(dburl, options=None)[source]

Initialize the database core

Parameters:
  • dburl (str) – Database url used to configure backend connection
  • options (geomancer.backend.settings.Config, optional) – Specify configuration for interacting with the database backend. Auto-detected if not set.
get_engine()[source]

Get the engine from the DBCore and load spatialite

load(df, index_label=None, index=False, if_exists='replace', **kwargs)[source]

Upload a pandas.DataFrame inside SQLite with a unique 32-char ID