Raster and Document Processing Utilities
PDF conversion, lossless GeoTIFF compression, and tested Rasterio layer packaging for two AWS Lambda CPU architectures.
Why it matters
A smaller raster is useful for analysis only if its numerical values survive compression. These tools separate analytical files from display previews and exercise packaged native dependencies in the runtime images that will load them.
My contribution
I wrote and published PDF conversion utilities and raster tooling that builds Lambda layer archives, checks them inside matching Lambda base images, and separates lossless GeoTIFF compression from lossy display previews.
Result
The tools provide configurable PDF conversion, lossless GeoTIFF compression, and separate opt-in JPEG previews. Rasterio layer archives for x86_64 and arm64 perform GeoTIFF read/write and coordinate transformation inside matching Lambda base images.
Tools
- Python
- Rasterio
- GDAL
- NumPy
- Docker
- AWS Lambda layer packaging
- pytest
- GitHub Actions
- PDF processing
Container checks establish packaging and native-library behavior for the tested runtime; no deployed AWS Lambda invocation is evidenced. Compression does not copy all source masks and ancillary metadata. JPEG previews also change original pixel values and must not replace analytical rasters.
A layer archive that is actually exercised
The build uses the official Lambda Python base image, installs Rasterio and NumPy under python/, and bundles libexpat under lib/. Packaging removes test directories, caches, and build-time console scripts. A separate clean container extracts the finished archive and loads only its Python packages for the smoke check.
- Both builds loaded Rasterio 1.4.4, GDAL 3.10.3, and NumPy 2.2.6.
- The smoke check writes and reads a generated GeoTIFF, verifies pixels and its coordinate reference system, and transforms a point through the bundled geospatial stack.
- Archive structure and size checks run before CI accepts the artifact. Runtime version and CPU architecture must match the consuming function.
| Architecture | ZIP size | Extracted size | Container smoke check |
|---|---|---|---|
| x86_64 | 50.8 MB | 163.0 MB | Passed |
| arm64 | 47.1 MB | 157.1 MB | Passed |
Separate analysis data from display previews
The command-line tool rejects identical input and output folders, skips existing outputs unless overwrite is requested, writes through temporary files, and returns a failure status if any input fails. Twenty-seven tests cover generated rasters and packaging fixtures without requiring sample imagery or Docker.
Lossless tests verify pixel values, nodata, coordinate reference system, and transform. Other source masks, scales, offsets, tags, and band metadata need separate handling. JPEG validity is derived from nodata and finite pixel values, rather than every possible source mask.
| Mode | Behavior | Intended use |
|---|---|---|
| Lossless, default | DEFLATE with a dtype-appropriate predictor; pixels written in source windows | Preserve numerical pixel values for analysis |
| JPEG, opt-in | Stretch valid values in each band to 8-bit and store a validity mask | Smaller visual previews; original values are lost |
Document preparation
The separate PDF-to-image utility remains available for selecting pages and controlling output format, resolution, grayscale conversion, and size. Its existing source link is retained; the October 1 update reviewed the raster repository.
Validation
The October 1 raster CI run passed 27 unit tests and built both Python 3.12 layer archives. Each archive passed an isolated GeoTIFF write/read and coordinate-transformation check in its matching Lambda base image.
Technical details
Independent portfolio tools · Raster tooling updated October 2026
Raster preparation must preserve analytical values, and native geospatial dependencies must match the runtime that loads them.
- Convert selected PDF pages with format, DPI, grayscale, and size options.
- Package Rasterio, its bundled GDAL, NumPy, and the required native library in a matching Lambda base image.
- Build and smoke-test Python 3.12 layers for x86_64 and arm64 in CI.
- Keep DEFLATE compression as the default and require an explicit JPEG mode for 8-bit display previews.