| Commit message (Collapse) | Author | Age | Files |
|---|
| |
|
|
|
| |
Using the default 0022 yields lock files with g-w, so trying to flock(2)
from a different user failed.
|
| |
|
|
|
| |
Cf. for instance svk:transmissionsnatsprojekt. It has an import
definition, but no source path/URL for webmap-download to operate on.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
modification time.
That way we can avoid the expensive unpack+import when the source
file(s) have not been updated since the last run. The check can be
bypassed with a new flag `--force`.
We use a sequence for the FID:s (primary key) and a UNIQUE constraint on
triplets (source_path, archive_member, layername) as GDAL doesn't
support multicolumns primary keys.
To avoid races between the stat(2) calls, gdal.OpenEx() and updates via
`webmap-download` runs we place a shared lock on the downloaded files.
One could resort to some tricks to eliminate the race between the first
two, but there is also some value in having consistency during the
entire execution of the script (a single source file can be used by
multiple layers for instance, and it makes sense to use the very same
file for all layers in that case).
We also intersperse dso.FlushCache() calls between _importSource() calls
in order to force the PG driver to call EndCopy() to detect errors and
trigger a rollback when _importSource() fails.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This avoid duplications when the same source file is used multiple times
(either by the same layer or by multiple layers). This change breaks
webmap-import, but that one will be refactored shortly.
It also breaks webmap-import-mrr.py, which is no longer used since
mineralrattigheter.zip can be downloaded from SGU's site directly.
|
| | |
|
| | |
|
| |
|
|
| |
This enables proper filtering by level etc. (incl. journald coloring).
|
| |
|
|
|
| |
We'll need that for layer creation (description, fields, creation
options, etc.).
|
| | |
|
| | |
|
| |
|
|
| |
Instead of sha1_hex().
|
| |
|
|
|
|
|
| |
passed only once.
This makes --debug less verbose by default and is useful for modules
that do lots of HTTP connections such as WMS probing.
|
| | |
|
| | |
|
| |
|
|
|
| |
We most likely only support POSIX systems, but best to hard code the
dependency.
|
| | |
|
| | |
|
| |
|
|
| |
dest needs to be stringified before the membership check.
|
| | |
|
| |
|
|
| |
This can be turned off with --no-exit-code.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This costs an extra file descriptor when --cachedir is unsed, but
os.link(f'/proc/self/fd/{fp.fileno()}', newpath,
dst_dir_fd=None, follow_symlinks=True)
calls link("/proc/self/fd/…", newpath) rather than
linkat(AT_FDCWD, "/proc/self/fd/…", AT_FDCWD, newpath, AT_SYMLINK_FOLLOW)
and the former fails with EXDEV (Invalid cross-device link).
|
| |
|
|
|
| |
Instead of using a dedicated executable. There is too much code
duplicate otherwise.
|
| |
|
|
|
| |
Don't use a lockdir by default, and always use lockfiles of the form
‘sha1_hex(path).lck’ if --lockdir is set.
|
| | |
|
| |
|
