|  | Commit message (Collapse) | Author | Age | Files | 
|---|
| ... |  | 
| | 
| 
| 
| 
| | There are a couple (15) of NULL AnmaldHa's, all from Sep 1, 2005 in
Borås, so we don't care about these. | 
| | |  | 
| | 
| 
| 
| | And set them to NULL. | 
| | 
| 
| 
| 
| | The previous default map was [-1] * n i.e., all source fields were
ignored. | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| | Path().joinpath('config.yml').exists() raises an exception when the
process doesn't have permission to traverse the current directory.  This
is the case, in particular, when running webmap-* as another user via
sudo while in the “normal” home directory. | 
| | |  | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| | On the desktop's terminal window (systemd 256) $SYSTEMD_EXEC_PID exists
and points at the login(1) process, so checking $JOURNAL_STREAM is
needed to decide whether the process runs from a .service file or not.
This is due to an incomplete logic in
42cfe748e23bad6fc51b14a3e5896a77fdb5c1d5 | 
| | 
| 
| 
| 
| 
| | Not sure why I thought there was duplicates but I must have done
something wrong as there are none in the 667034+223463+214082+
72656 = 1177234 features found after removing the spatial filter. | 
| | 
| 
| 
| | This enables proper filtering by level etc. (incl. journald coloring). | 
| | 
| 
| 
| 
| 
| 
| 
| | This is useful to bootstrap the database without having to run
	GRANT CREATE ON SCHEMA postgis TO webmap_import;
after DROPping the database or re-creating the cluster. | 
| | 
| 
| 
| | (Commented out for now since Bookworm has only GDAL v3.6.) | 
| | 
| 
| 
| 
| 
| 
| | Tighten column width and replace fixed-width numerics with single
precisions floats.
Also, align column names with the documented ones. | 
| | 
| 
| 
| 
| 
| | Despite using gdal.UseExceptions() a failed call doesn't raise an
exception, so we need to check the return value to avoid missing
features. | 
| | |  | 
| | |  | 
| | |  | 
| | |  | 
| | |  | 
| | 
| 
| 
| 
| 
| | The extent is expressed in config['SRS'] in traditional GIS order
(easting/northing ordered: minX, minY, maxX, maxY), but the destination
layers might be pre-existing and use other SRS:es or mapping strategy. | 
| | |  | 
| | 
| 
| 
| 
| 
| | The configured extent is always expressed in the destination SRS, so it
needs to be transformed into the source SRS.  Like apps/ogr2ogr_lib.cpp,
we segmentize it to make sure it is sufficiently densified. | 
| | 
| 
| 
| | (Commented out in config.yml for now since Bookworm has only v3.6.) | 
| | |  | 
| | 
| 
| 
| | And getFieldSubTypeCode() to parseSubFieldType(). | 
| | 
| 
| 
| 
| 
| | We're TRUNCATE'ing the output layers (tables) at every run and aren't
reseting the sequences so they would soon overflow 32-bits FIDs
("sks:UtfordAvverk" has almost 300k features within the extent…). | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | There is still a few things to do (such as reprojection and geometry
changes) but it's mostly working.
We roll out our own ogr2ogr/GDALVectorTranslate()-like function version
because GDALVectorTranslate() insists in calling StartTransaction()
https://github.com/OSGeo/gdal/issues/3403 while we want a single
transaction for the entire desination layer, including truncation,
source imports, and metadata changes.
Surprisingly our version is not much slower than the C++ one.  Importing
the 157446 (of 667034) features from sksUtfordAvverk-2000-2015.shp takes
14.3s while
	ogr2ogr -f PostgreSQL \
	    -doo ACTIVE_SCHEMA=postgis \
	    --config PG_USE_COPY YES \
	    --config OGR_TRUNCATE YES \
	    -append \
	    -fieldmap "0,-1,-1,-1,-1,1,2,3,4,5,6,7,8,9,10,11,12,13" \
	    -nlt MULTIPOLYGON -nlt PROMOTE_TO_MULTI \
	    -gt unlimited \
	    -spat 110720 6927136 1159296 7975712 \
	    -nln "sks:UtfordAvverk" \
	    PG:"dbname='webmap' user='webmap_import'" \
	    /tmp/x/sksUtfordAvverk-2000-2015.shp \
	    sksUtfordAvverk-2000-2015
takes 14s.
Merely opening /tmp/x/sksUtfordAvverk-2000-2015.shp and looping through
its (extent-filtered) features results in a runtime of 4.3s. | 
| | 
| 
| 
| 
| | We'll need that for layer creation (description, fields, creation
options, etc.). | 
| | 
| 
| 
| | The configured extent is always relative to the configured SRS. | 
| | |  | 
| | |  | 
| | |  | 
| | 
| 
| 
| 
| | We always exit after a single WMS request if the layer has no SVG paths.
Don't print the request count in that case. | 
| | |  | 
| | |  | 
| | |  | 
| | 
| 
| 
| | Instead of sha1_hex(). | 
| | 
| 
| 
| 
| 
| 
| | passed only once.
This makes --debug less verbose by default and is useful for modules
that do lots of HTTP connections such as WMS probing. | 
| | 
| 
| 
| | (Incl. failure.) | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Unfortunately SGU/Bergsstaten doesn't offer layer files to download, but
it has an online webmap (WMS) at https://apps.sgu.se/kartvisare/kartvisare-mineralrattigheter.html
so we add a dedicated module to probe and fetch features from it.
Double checked that the resulting combination of GeoJSON files does not
result in data loss compared to the previous (private) script:
    sort_features() { jq -S '.features |= sort_by(.properties.Name, .properties.Layer, .properties.Area)' }
    diff -u --color=auto \
            <(ogr2ogr -f GeoJSON -lco COORDINATE_PRECISION=2 /vsistdout/ $dir/Mineralrättigheter.gpkg \
                SE.GOV.SGU.MRR.BEARBETNINGSKONCESSIONER_APPLIED_VY | jq '.name = "MRR:" + .name' | sort_features) \
            <(ogr2ogr -f GeoJSON -lco COORDINATE_PRECISION=2 -nlt MULTIPOLYGON -nlt PROMOTE_TO_MULTI \
                /vsistdout/ mrr/bearbetningskoncessioner_applied.geojson | sort_features)
(and similar for other layers). | 
| | |  | 
| | |  | 
| | 
| 
| 
| 
| | We most likely only support POSIX systems, but best to hard code the
dependency. | 
| | |  | 
| | |  | 
| | |  | 
| | |  | 
| | 
| 
| 
| | dest needs to be stringified before the membership check. | 
| | |  |