aboutsummaryrefslogtreecommitdiffstats
path: root/config.yml
Commit message (Collapse)AuthorAgeFiles
* Add layer definitions for Vindbrukskollen layers.Guilhem Moulin2024-06-221
| | | | | | | | | | | | | | | | | Only the land-based ones for now. Source: https://www.energimyndigheten.se/energisystem-och-analys/elproduktion/vindkraft/vindbrukskollen/ Layers: - https://ext-geodatakatalog.lansstyrelsen.se/GeodataKatalogen/srv/api/records/GetMetaDataById?id=ed5814b2-08bf-493a-a164-7819e1b590d6 LST Vindbrukskollen landbaserade vindkraftverk - https://ext-geodatakatalog.lansstyrelsen.se/GeodataKatalogen/srv/api/records/GetMetaDataById?id=c816bd1e-bc6c-487f-a962-770f05f677b6 LST Vindbrukskollen landbaserade projekteringsområden - https://ext-geodatakatalog.lansstyrelsen.se/GeodataKatalogen/srv/api/records/GetMetaDataById?id=c290bc31-1af8-497e-a9a5-87fcec55d0ce LST Vindbrukskollen havsbaserad vindkraft Webmaps: - https://vbk.lansstyrelsen.se/ - https://ext-geodatakatalog.lansstyrelsen.se/GeodataKatalogen/srv/swe/catalog.search#/map
* Conditionally use GetTZFlag()/SetTZFlag() depending on the GDAL version.Guilhem Moulin2024-06-201
| | | | | | | | | OGRFieldDefn: add GetComment() / SetComment() methods were added in OGR 3.8.0, cf. https://github.com/OSGeo/gdal/blob/master/NEWS.md#core-3 . Don't comment out TZ on field definitions. Instead we check the GDAL/OGR version and ignore TZ on field definitions if the OGR version is too old.
* Conditionally use GetComment()/SetComment() depending on the GDAL version.Guilhem Moulin2024-06-201
| | | | | | | | | OGRFieldDefn: add GetComment() / SetComment() methods were added in OGR 3.7.0, cf. https://github.com/OSGeo/gdal/blob/master/NEWS.md#core-5 . Don't comment out comments on field definitions. Instead we check the GDAL/OGR version and ignore comments on field definitions if the OGR version is too old.
* config.yml: Start comments with an uppercase.Guilhem Moulin2024-06-201
|
* Add logic for field regex substitution.Guilhem Moulin2024-06-191
| | | | | | | | | This is useful to replace a YYYYMMDD formatted date with YYYY-MM-DD. The target field can then be set to not-nullable and its type set to Date, as the OGR_F_SetField*() with take care of the conversion. We could also do that via an SQL query, but in our case the sources are not proper RDBMS so SQL is emulated anyway.
* sks:UtfordAvverk: Add NOT NULL constraint on AnmaldHa column.Guilhem Moulin2024-06-191
| | | | | There are a couple (15) of NULL AnmaldHa's, all from Sep 1, 2005 in Borås, so we don't care about these.
* Add logic to replace field value literals.Guilhem Moulin2024-06-191
| | | | And set them to NULL.
* webmap-import: Rename ‘fields’ list/dict to ‘field-map’.Guilhem Moulin2024-06-161
|
* Add schema for MRR layers.Guilhem Moulin2024-06-121
|
* sks:UtfordAvverk: Add UNIQUE constraint on OBJECTID.Guilhem Moulin2024-06-121
| | | | | | Not sure why I thought there was duplicates but I must have done something wrong as there are none in the 667034+223463+214082+ 72656 = 1177234 features found after removing the spatial filter.
* config.yml: Add field comments.Guilhem Moulin2024-06-111
| | | | (Commented out for now since Bookworm has only GDAL v3.6.)
* config.yml: Refine destination schema.Guilhem Moulin2024-06-111
| | | | | | | Tighten column width and replace fixed-width numerics with single precisions floats. Also, align column names with the documented ones.
* config.yml: Improve description.Guilhem Moulin2024-06-111
|
* config.yml: Preserve layer name from administrativindelning.gpkg.Guilhem Moulin2024-06-111
|
* Fix extent logic when the SRS of the output layer is not the destination SRS.Guilhem Moulin2024-06-111
| | | | | | The extent is expressed in config['SRS'] in traditional GIS order (easting/northing ordered: minX, minY, maxX, maxY), but the destination layers might be pre-existing and use other SRS:es or mapping strategy.
* Add TZFlag support (for GDAl ≥3.8).Guilhem Moulin2024-06-101
| | | | (Commented out in config.yml for now since Bookworm has only v3.6.)
* config.yml: Set layer creation option FID64=YES.Guilhem Moulin2024-06-101
| | | | | | We're TRUNCATE'ing the output layers (tables) at every run and aren't reseting the sequences so they would soon overflow 32-bits FIDs ("sks:UtfordAvverk" has almost 300k features within the extent…).
* Add `webmap-import` script to import source layers.Guilhem Moulin2024-06-101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is still a few things to do (such as reprojection and geometry changes) but it's mostly working. We roll out our own ogr2ogr/GDALVectorTranslate()-like function version because GDALVectorTranslate() insists in calling StartTransaction() https://github.com/OSGeo/gdal/issues/3403 while we want a single transaction for the entire desination layer, including truncation, source imports, and metadata changes. Surprisingly our version is not much slower than the C++ one. Importing the 157446 (of 667034) features from sksUtfordAvverk-2000-2015.shp takes 14.3s while ogr2ogr -f PostgreSQL \ -doo ACTIVE_SCHEMA=postgis \ --config PG_USE_COPY YES \ --config OGR_TRUNCATE YES \ -append \ -fieldmap "0,-1,-1,-1,-1,1,2,3,4,5,6,7,8,9,10,11,12,13" \ -nlt MULTIPOLYGON -nlt PROMOTE_TO_MULTI \ -gt unlimited \ -spat 110720 6927136 1159296 7975712 \ -nln "sks:UtfordAvverk" \ PG:"dbname='webmap' user='webmap_import'" \ /tmp/x/sksUtfordAvverk-2000-2015.shp \ sksUtfordAvverk-2000-2015 takes 14s. Merely opening /tmp/x/sksUtfordAvverk-2000-2015.shp and looping through its (extent-filtered) features results in a runtime of 4.3s.
* config.yml: Allow configuration setting at the layer root.Guilhem Moulin2024-06-091
| | | | | We'll need that for layer creation (description, fields, creation options, etc.).
* Add layer definitions for Mineralrättigheter.Guilhem Moulin2024-06-031
| | | | | | | | | | | | | | | | | | Unfortunately SGU/Bergsstaten doesn't offer layer files to download, but it has an online webmap (WMS) at https://apps.sgu.se/kartvisare/kartvisare-mineralrattigheter.html so we add a dedicated module to probe and fetch features from it. Double checked that the resulting combination of GeoJSON files does not result in data loss compared to the previous (private) script: sort_features() { jq -S '.features |= sort_by(.properties.Name, .properties.Layer, .properties.Area)' } diff -u --color=auto \ <(ogr2ogr -f GeoJSON -lco COORDINATE_PRECISION=2 /vsistdout/ $dir/Mineralrättigheter.gpkg \ SE.GOV.SGU.MRR.BEARBETNINGSKONCESSIONER_APPLIED_VY | jq '.name = "MRR:" + .name' | sort_features) \ <(ogr2ogr -f GeoJSON -lco COORDINATE_PRECISION=2 -nlt MULTIPOLYGON -nlt PROMOTE_TO_MULTI \ /vsistdout/ mrr/bearbetningskoncessioner_applied.geojson | sort_features) (and similar for other layers).
* Rename script into module and run module.download().Guilhem Moulin2024-06-021
| | | | | Instead of using a dedicated executable. There is too much code duplicate otherwise.
* Add layer definitions from Sametinget and Vindbrukskollen.Guilhem Moulin2024-06-011
|
* Add `webmap-download` script.Guilhem Moulin2024-06-011
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-16 19:17:13 +0000