aboutsummaryrefslogtreecommitdiffstats
path: root/common.py
Commit message (Collapse)AuthorAgeFiles
* Add a simple CGI to retrieve features from PostgreSQL/PostGIS.Guilhem Moulin12 days1
| | | | | | | | | | | This avoids exporting features properties and geometries as GeoJSON. (We could do the same for tiles, but it would be too slow.) The downside is that the tables can't be read from while `webmap-import` is running, so metadata lookup might briefly timeout during the daily update. It's still worth doing as it greatly simplifies deployment and saves GiB of disk space.
* webmap-import: Add a cache layer and store the source file's last ↵Guilhem Moulin2025-04-231
| | | | | | | | | | | | | | | | | | | | | | | | modification time. That way we can avoid the expensive unpack+import when the source file(s) have not been updated since the last run. The check can be bypassed with a new flag `--force`. We use a sequence for the FID:s (primary key) and a UNIQUE constraint on triplets (source_path, archive_member, layername) as GDAL doesn't support multicolumns primary keys. To avoid races between the stat(2) calls, gdal.OpenEx() and updates via `webmap-download` runs we place a shared lock on the downloaded files. One could resort to some tricks to eliminate the race between the first two, but there is also some value in having consistency during the entire execution of the script (a single source file can be used by multiple layers for instance, and it makes sense to use the very same file for all layers in that case). We also intersperse dso.FlushCache() calls between _importSource() calls in order to force the PG driver to call EndCopy() to detect errors and trigger a rollback when _importSource() fails.
* webmap-import: Break down into separate modules.Guilhem Moulin2025-04-211
|
* webmap-import: Major refactoring.Guilhem Moulin2025-04-191
|
* Factor sources in config.yml.Guilhem Moulin2025-04-191
| | | | | | | | | This avoid duplications when the same source file is used multiple times (either by the same layer or by multiple layers). This change breaks webmap-import, but that one will be refactored shortly. It also breaks webmap-import-mrr.py, which is no longer used since mineralrattigheter.zip can be downloaded from SGU's site directly.
* Add type hints and refactor a bit to please pylint.Guilhem Moulin2025-04-191
|
* Fix common.format_time().Guilhem Moulin2025-04-171
| | | | And add an optional precision argument
* Add `webmap-publish` script to export layers to Mapbox Vector Tiles.Guilhem Moulin2024-09-251
|
* common.py: Add a check for $JOURNAL_STREAM.Guilhem Moulin2024-09-191
| | | | | | | | | | $JOURNAL_STREAM is set when the WM is started with `systemd-cat --identifier="sway" sway` or similar. So instead we also check that $SYSTEMD_EXEC_PID points at the current PID. This is due to an incomplete logic in d40c908f9eb768c3f93ae7dc60ee7eb4c358ed4f and 42cfe748e23bad6fc51b14a3e5896a77fdb5c1d5.
* Conditionally use GetComment()/SetComment() depending on the GDAL version.Guilhem Moulin2024-06-201
| | | | | | | | | OGRFieldDefn: add GetComment() / SetComment() methods were added in OGR 3.7.0, cf. https://github.com/OSGeo/gdal/blob/master/NEWS.md#core-5 . Don't comment out comments on field definitions. Instead we check the GDAL/OGR version and ignore comments on field definitions if the OGR version is too old.
* common.load_config(): Don't fail when curdir is not traversable.Guilhem Moulin2024-06-121
| | | | | | | Path().joinpath('config.yml').exists() raises an exception when the process doesn't have permission to traverse the current directory. This is the case, in particular, when running webmap-* as another user via sudo while in the “normal” home directory.
* common.py: Add a check for $JOURNAL_STREAM.Guilhem Moulin2024-06-121
| | | | | | | | | On the desktop's terminal window (systemd 256) $SYSTEMD_EXEC_PID exists and points at the login(1) process, so checking $JOURNAL_STREAM is needed to decide whether the process runs from a .service file or not. This is due to an incomplete logic in 42cfe748e23bad6fc51b14a3e5896a77fdb5c1d5
* Use systemd.journal to log to journald when sarted via .service files.Guilhem Moulin2024-06-111
| | | | This enables proper filtering by level etc. (incl. journald coloring).
* config.yml: Allow configuration setting at the layer root.Guilhem Moulin2024-06-091
| | | | | We'll need that for layer creation (description, fields, creation options, etc.).
* load_config(): Fail if an extent is set while no SRS is.Guilhem Moulin2024-06-081
| | | | The configured extent is always relative to the configured SRS.
* webmap-download*: Use narrow non-breaking space (U+202F) as unit separator.Guilhem Moulin2024-06-081
|
* Add GPL-3+ license headers.Guilhem Moulin2024-06-031
|
* webmap-download: Move format_bytes() and format_time() to common.py.Guilhem Moulin2024-06-031
|
* Add comment about dir_fd-aware functions.Guilhem Moulin2024-06-031
|
* Rename script into module and run module.download().Guilhem Moulin2024-06-021
| | | | | Instead of using a dedicated executable. There is too much code duplicate otherwise.
* Add `webmap-download` script.Guilhem Moulin2024-06-011