pycmac package¶

Module contents¶

pycmac.orientation module¶

Ciaran Robb

A module which calls Micmac image orientation commands whilst providing additional file parsing and sorting operations.

orientation.bundle_adjust(folder, algo='Fraser', proj='30 +north', ext='JPG', calib=None, gpsAcc='1', sep=',', gcp=None, gcpAcc=['0.03', '1'], meshlab=False, useGps=True)¶

A function running the relative orientation/bundle adjustment with micmac

A calibration subset is optional

Notes

Underlying cmds include

(Tapas, centrebascule, Campari, ChgSysCo, OriExport)

Pleae note, that if GCPs are used, then the on-board GPS will not be included in the final bundle adjustment as the on-board will not have positive effect on the overall result. This assumes your GCPs come from a DGPS, otherwise don’t use them!

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
calib (string) – a calibration subset (optional - otherwise the martini initialisation will be used)
ext (string) – image extention e.g JPG, tif
gpsAcc (string) – an estimate in metres of the onboard GPS accuracy
gcp (string) – whether to process gcps - you MUST have a GCP file in the MM format of #F=N X Y Z and MUST be in the working dir
gcpAcc (list (of strings)) – an estimate of the GCP measurment accuarcy [on the ground in metres, in pixels]
exif (bool) – if the GPS info is embedded in the image exif check this as True to convert back to geographic coordinates, If previous steps always used a csv for img coords ignore this
useGps (bool) – if the GPS info is untrustyworthy with a lot of the data (eg Dji Phantom - z) simply transform from rel to ref coordinate sys without GPS aided bundle adjust.

orientation.feature_match(folder, csv=None, proj='30 +north', method='File', resize=None, ext='JPG', delim=' ', schnaps=True, dist=None)¶

A function running the feature detection and matching with micmac

Notes

Underlying cmds include

(Tapioca and Schnapps)

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
csv (string) – a path to the csv of GPS coords
resize (int) – The long axis in pixels to optionally resize the imagery
ext (string) – image extention e.g JPG, tif
dist (string) – distance for nearest neighbour search

orientation.gps_orient(folder, algo='Fraser', proj='30 +north', ext='JPG', gpsAcc='1', meshlab=False)¶

A function running the gps bundle adjustment with micmac

Notes

Underlying cmds include

(Campari, AperiCloud)

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
ext (string) – image extention e.g JPG, tif
gpsAcc (string) – an estimate in metres of the onboard GPS accuracy
useGps (bool) – if the GPS info is untrustyworthy with a lot of the data (eg Dji Phantom - z) simply transform from rel to ref coordinate sys without GPS aided bundle adjust.

orientation.rel_orient(folder, algo='Fraser', proj='30 +north', ext='JPG', calib=None, sep=', ', meshlab=False, useGps=True)¶

A function running the relative orientation with micmac

A calibration subset is optional

Notes

Underlying cmds include

(Tapas, centrebascule)

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
calib (string) – a calibration subset (optional - otherwise the martini initialisation will be used)
ext (string) – image extention e.g JPG, tif

pycmac.dense_match module¶

Ciaran Robb

A module which calls Micmac dense matching commands and mosaicking operations, whilst a providing dditional file parsing, sorting, image masking and multi-band functionallity.

dense_match.c3dc(folder, mode='Statue', ext='JPG', orientation='Ground_UTM', sub=None, DefCor='0', delim=', ', **kwargs)¶

A function calling the Culture 3D-cloud function

Notes

see MicMac tools link for further possible args - just put the module cmd as a kwarg The kwargs must be exactly the same case as the mm3d cmd options

Parameters

folder (string) – working directory
proj (string) – a proj4/gdal like projection information e.g ESPG:32360
mode (string) – Correlation mode - MicMac, BigMac, QuickMac, Forest, Statue Default is BigMac
ext (string) – image extention e.g JPG, tif
orientation (string) – orientation folder to use (generated by previous tools/cmds) default is “Ground_UTM”

dense_match.dense_pcl(folder, mode='PIMs', out='psm.ply')¶

A function for passing a subset to Malt or PIMS

Parameters

folder (string) – working directory
mode (string) – The micmac processing mode Malt or PIMs
out (string) – The output point cloud in .ply format

dense_match.feather(folder, proj='30 +north', mode='PIMs', ms=['r', 'g', 'b'], Dist='100', ApplyRE='1', ComputeRE='1', subset=None, outMosaic=None, rmtile=False, mp=False, Label=False, redo=False, delim=',', **kwargs)¶

A function calling the TestLib SeamlineFeathering command for use in python

Notes

The native micmac function only processes single band - this function provides a multi band capability

An example usage:

feather(folder, mode=’Malt’,outMosaic=”Feathermp.tif”, Dist=”100”, ApplyRE=”1”, ComputeRE=”1”, mp=True)

Parameters

folder (string) – working directory
proj (string) – a proj4/gdal like projection information e.g “ESPG:32360”
mode (string) – Ortho folder use either PIMs or Malt here
ms (list) – if a multi band image the band names in a list
mp (int) – Whether to employ multiprocessing
Dist (string) – The chamfer distance to feather, def None
Lambda (string) – lambda value for gaussian distance weighting, def 0.4
ApplyRE (string) – whether to apply radiometric equalisation, def 1
ComputeRE (string) – whether to compute radiometric equalisation
subset (string) – the absolute path to a csv defining a subset list of images to process if left as None, all the images will be mosiacked.
SzBox (string) – size of processing block, def [25000,25000]
SzTile (string) – size of processing block, def [25000,25000]
Buffer (string) – Buffer [pix] to apply for each tile in order to avoid edge effect, def=300

Label: bool: Whether to use a tawny generated label map (Label.tif) This requires a run of Tawny on the Ortho folder first!
redo: bool: To rerun without having to copy the band folders on big datasets Useful as this step is time consuming during processing otherwise
delim: string: If using a csv-based subset list, the delimeter can be specified- it is comma by default

dense_match.malt(folder, proj='30 +north', mode='Ortho', ext='JPG', orientation='Ground_UTM', DoOrtho='1', DoMEC='1', DefCor='0', sub=None, delim=', ', ResolTerrain=None, BoxTerrain=None, mask=None, **kwargs)¶

A function calling the Malt command for use in python.

Upon successful completion, this will return the directory OUTPUT with DSM, ortho (if desried) and correlation image

Notes

see MicMac tools link for further possible kwargs - just put the module cmd as a kwarg The kwargs must be exactly the same case as the mm3d cmd options e.g = UseGpu=’1’

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
mode (string) – Correlation mode - Ortho, UrbanMNE, GeomImage
ext (string) – image extention e.g JPG, tif
orientation (string) – orientation folder to use (generated by previous tools/cmds) default is “Ground_UTM”
sub (string) – path to csv containing an image subset
BoxTerrain (list) – The bottom left and top right coordinates of a bounding box to constrain processing e.g. [480644.0,5752033.4,481029.4,5752251.6]
mask (string) – path to a polygon file (eg shape, geojson) that will yield a bounding box to constrain processing

dense_match.malt_batch(folder, mode='Ortho', mp=-1, gp=0, window=2, mx=None, ext='JPG', tiles='3, 3', overlap=10, bb=False)¶

A function for passing a subset to Malt or PIMS

Parameters

folder (string) – working directory
mode (string) – The micmac processing mode eg ‘Ortho’ (default)
gp (int (optional)) – path to gpu suppoted micmac bin folder
mp (int) – Correlation mode - Ortho, UrbanMNE, GeomImage
ext (string) – image extention e.g JPG, tif
orientation (string) – orientation folder to use (generated by previous tools/cmds) default is “Ground_UTM”

dense_match.ossimmosaic(folder, proj='30 +north', mode='ossimFeatherMosaic', nt=-1, rmtile=False)¶

A function mosaicing using the ossim library

Notes

Parameters

folder (string) – working directory - likely a malt-ortho or pims equiv
proj (string) – a proj4/gdal like projection information
mode (string) – ossim mosaic type of the options below: ossimBlendMosaic ossimMaxMosaic ossimImageMosaic ossimClosestToCenterCombiner ossimBandMergeSource ossimFeatherMosaic
rmtile (bool) – remove every other tile to perhaps improve things

dense_match.pims(folder, mode='BigMac', ext='JPG', orientation='Ground_UTM', DefCor='0', sub=None, delim=', ', mask=False, **kwargs)¶

A function calling the PIMs command for use in python

Notes

see MicMac tools link for further possible args - just put the module cmd as a kwarg The kwargs must be exactly the same case as the mm3d cmd options

Parameters

folder (string) – working directory
proj (string) – a proj4/gdal like projection information e.g ESPG:32360
mode (string) – Correlation mode - MicMac, BigMac, QuickMac, Forest, Statue Default is BigMac
ext (string) – image extention e.g JPG, tif
orientation (string) – orientation folder to use (generated by previous tools/cmds) default is “Ground_UTM”
mask (bool) – image extention e.g JPG, tif

dense_match.pims2mnt(folder, proj='30 +north', mode='BigMac', DoOrtho='1', DoMnt='1', **kwargs)¶

A function calling the PIMs2MNT command for use in python

Notes

see MicMac tools link for further possible args - just put the module cmd as a kwarg The kwargs must be exactly the same case as the mm3d cmd options

Parameters

folder (string) – working directory
proj (string) – a proj4/gdal like projection information e.g “ESPG:32360”
mode (string) – Correlation folder to grid/rectify - MicMac, BigMac, QuickMac, Forest, Statue Default is BigMac

dense_match.tawny(folder, proj='30 +north', mode='PIMs', Out=None, rmtile=False, **kwargs)¶

A function calling the Tawny command for use in python

Notes

see MicMac tools link for further possible args - just put the module arg as a keyword arg The kwargs must be exactly the same case as the mm3d cmd options

If struggling with even illumination, try:

tawny(folder, proj=”30 +north”, mode=’PIMs’, Out=None, rmtile=False,: DEq=”1”, DegRapXY=”[4,4]” SzV=”75” NbPerIm=”5e4”)

Parameters

folder (string) – working directory
proj (string) – a proj4/gdal like projection information e.g “ESPG:32360”
mode (string) – Either Malt or PIMs depending on the previous process
rmtile (bool) – Remove every second tile - can speed up and sometimes improve even illumination across mosaic

pycmac.utilities module¶

Ciaran Robb

A module which provides various ancillary functions for processing SfM with Micmac.

utilities.array2raster(array, bands, inRaster, outRas, dtype, FMT=None)¶

Save a raster from a numpy array using the geoinfo from another.

Parameters

array (np array) – a numpy array.
bands (int) – the no of bands.
inRaster (string) – the path of a raster.
outRas (string) – the path of the output raster.
dtype (int) – though you need to know what the number represents! a GDAL datatype (see the GDAL website) e.g gdal.GDT_Int32
FMT (string) – (optional) a GDAL raster format (see the GDAL website) eg Gtiff, HFA, KEA.

utilities.calib_subset(folder, csv, ext='JPG', algo='Fraser', delim=', ')¶

A function for calibrating on an image subset then initialising a global orientation

Notes

Eliminates the need to type lenghty file patterns via Micmac and combines

the subset and main orientation in one function

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
mode (string) – Correlation mode - Ortho, UrbanMNE, GeomImage
ext (string) – image extention e.g JPG, tif
orientation (string) – orientation folder to use (generated by previous tools/cmds) default is “Ground_UTM”

utilities.convert_c3p(folder, lognm, ext='JPG', mspec=False, delim=', ')¶

Edit csv file for c3p to work with MicMac.

This is intended for the output from the software of a C-Astral drone

This assumes the column order is name, x, y, z

Parameters

folder (string) – path to folder containing jpegs
lognm (string) – path to c3p derived csv file
ext (string) – image extension
delim (string) – the csv delimiter of the input

utilities.detect_blur(inFolder, ext='JPG', threshold=100)¶

Detect if images are blurry then move them to a blur folder prior to SfM

using a laplacian convolution…..

Parameters

inFolder (string) – the input folder with images
ext (string) – image extention e.g JPG, tif
threshold (int) – the threshold of blurryness to remove photo around 100 usually does it

utilities.get_vid(video, outFolder, ext='JPG', cam='Iphone_SE', foc='2.4', foc35='29')¶

Extract the frames from a video with the view to using them for SfM

The camera focal length and 35mm equiv must be known as the frames will not contain them

If you have taken a pic with the same platform use exiftool to obtain the info you need

Parameters

video (string) – the input raster
outFolder (string) – the masking value that delineates pixels to be kept
num (int) – the subdivision - e.g. every fifth image
ext (string) – image extention e.g JPG, tif
cam (string) – the camera focal length as it is required
cam (string) – the camera focal length as it is required

utilities.hist_match(inputImage, templateImage)¶

Adjust the pixel values of an image such that its histogram matches that of a target image.

Writes to the inputImage dataset so that it matches

As the entire band histogram is required this can become memory intensive with very big images

Inspire by/adapted from something on stack on image processing - credit to that author

Parameters

inputImage (string) – image to transform; the histogram is computed over the flattened array
templateImage (string) – template image can have different dimensions to source

utilities.make_csv(folder, ext='tif')¶

make a csv for use with micmac

Parameters

folder (string) – working directory
ext (string) – image extention e.g JPG, tif

utilities.make_sys_utm(folder, proj)¶

Make an xml for the geographic system in micmac

Parameters

folder (string) – working directory
proj (a proj format definition in UTM) – e.g +proj=utm +zone=30 +north +ellps=WGS84 +datum=WGS84 +units=m +no_defs”

utilities.make_xml(csvFile, folder, sep=' ')¶

Make an xml for the rtl system in micmac

Parameters: csvFile (string) – csv file with coords to use

utilities.mask_raster_multi(inputIm, mval=1, outval=None, mask=None, blocksize=256, FMT=None, dtype=None)¶

Perform a numpy masking operation on a raster where all values corresponding to mask value are retained - does this in blocks for efficiency on larger rasters

Parameters

inputIm (string) – the input raster
mval (int) – the masking value that delineates pixels to be kept
outval (numerical dtype eg int, float) – the areas removed will be written to this value default is 0
mask (string) – the mask raster to be used (optional)
FMT (string) – the output gdal format eg ‘Gtiff’, ‘KEA’, ‘HFA’

blocksizeint: the chunk of raster read in & write out

utilities.mm3d(folder, cmd, *args, **kwargs)¶

Execute a micmac command via mm3d just as it would be in the micmac command line - purely for convenience - you’d be as well using the command line directly

All args kwargs must be in this format where the arg is a string and the kwarg is a bytecode=string

eg - mm3d(myfolder, ‘Malt’, ‘Ortho’, “.*JPG” ‘Ground_UTM’, DefCor=”0”, DoOrtho=”0”)

There will be the odd time where this may not work!

Parameters

folder (string) – working directory
ext (string) – image extention e.g JPG, tif

utilities.mv_subset(csv, inFolder, outfolder, sep=' ')¶

Move a subset of images based on a MicMac csv file

Micmac csv format is column header #F=N X Y Z with space delimiters

Parameters

folder (string) – path to folder containing jpegs
lognm (string) – path to c3p derived csv file
sep (string) – the delimiter of the csv (space is default)

utilities.num_subset(inFolder, outFolder, num=5, ext='JPG')¶

Pick a subset of images from a video such as every fifth image (frame)

This assumes you have used ffmpeg or a similar application to extract the frames first

Parameters

inFolder (string) – the folder with the images in
outFolder (string) – the folder the output images will go
num (int) – the subdivision - e.g. every fifth image
ext (string) – image extention e.g JPG, tif

utilities.orbplot(folder, imgs)¶

Plot matching features across 3 images using ORB The images MUST overlap

Parameters

folder (string) – the working directory with images in
imgs (list of strings) – the images to plot feature matches in the format [‘img1’, ‘img2’, ‘img3’]

utilities.pims_mask(inShp, folder)¶

Make an xmlmask from a shapefile for polyg3d for pims/c3dc The shapefile MUST be rectilinear

Parameters: inShp (string) – shapefile with coords to use

utilities.plot_img_csv(folder, csv='log.csv', delim=' ', dispCol='Z')¶

Plot image GPS csv on a map with mapbox

Parameters

folder (string) – working directory
csv (string) – csv name - log.csv is default
delim (string) – delimiter of csv ” ” (space) is defaut;

utilities.plot_result(folder, inRas, proj=32630)¶

Plot image GPS csv on a map with mapbox

Parameters

folder (string) – working directory
inRas (string) – name of image in the OUTPUT folder

utilities.raster2array(inRas, bands=[1])¶

Read a raster and return an array, either single or multiband

Parameters

inRas (string) – input raster
bands (list) – a list of bands to return in the array

utilities.rmse_vector_lyr(inShape, attributes)¶

Using sklearn get the rmse of 2 vector attributes (the actual and predicted of course in the order [‘actual’, ‘pred’])

Parameters

inShape (string) – the input vector of OGR type
attributes (list) – a list of strings denoting the attributes

pycmac.mspec module¶

Ciaran Robb

https://github.com/Ciaran1981/Sfm

This module processes data from the micasense red-edge camera for use in MicMac or indeed other SfM software

This correction is based on the material on the micasense lib git site, though this uses as fork of the micasense lib with some alterations.

mspec.align_template(imAl, mx, reflFolder, rf, plots, warp_md)¶

mspec.clip_raster(inRas, inShape, outRas)¶

Clip a raster using the extent of a shapefile

Parameters

inRas (string) – the input image
inShape (string) – the input polygon file path
outRas (string (optional)) – the clipped raster
nodata_value (numerical (optional)) – self explanatory

mspec.mica_csv(folder, time_date=False)¶

mspec.mspec_proc(imgFolder, alIm, srFolder, precal=None, postcal=None, refBnd=4, nt=-1, mx=100, stk=1, plots=False, panel_ref=None, warp_type='MH')¶

A function processing the micasense data to surface reflectance with a choice of output types dependent on preference.

Notes

A set of RGB & RReNir images is recommended for processing with MicMac

Either 5 band stack or single band outputs can also be produced.

Parameters

imgFolder (string) – a directory containing the raw imagery
precal (string) – directory containing pre-flight calibration panels pics

alIm: string: 4 digit code of the image to align band images e.g. “0023”
srFolder: string: path to directory for surface reflectance imagery
postcal: string: directory containing pre-flight calibration panels pics
refBnd: int: The band to which all others are aligned def 4 works best
nt: int: No of threads to use
mx: int: Max iterations for alignment (uses opencv motion homography)
stk: int: The various multi-band stacking options 1 = A set of RGB & RReNir images (best for MicMac) None = Single band images in separate folders are returned

plots: bool

Whether to display plots of the alignment images for visual inspection

panel_ref: list

The panel ref values unique to your panel/camera If left as None, it will load the authors camera values!

warp_type: string

The type of warping used to align the bands from the following choice of cv2 modes

Either:
MH for MOTION_HOMOGRAPHY Affine for MOTION_AFFINE

mspec.slant_view_proc(folder, nt=-1)¶

mspec.stack_rasters(inRas1, inRas2, outRas, dtype=5, slantr=False)¶

Stack the intersected extent of £-band composites from MicMac which contain

RGB and RReNir respectively. Obviously the red band will not be written twice!

Parameters

inRas1 (string) – path to RGB image
inRas2 (string) – path to RReNir image
outRas3 (string) – path to outputted stack
dtype (int) – gdal datatype e.g. gdal.GDT_Int32 (default)

pycmac.sfm module¶

@author: ciaran robb

Complete Sfm piplines using the micmac lib.

sfm.mspec_sfm(folder, proj='30 +north', csv=None, sub=None, gpsAcc='1', gcp=None, gcpAcc=['0.03', '1'], sep=',', mode='PIMs', submode='Forest', ResolTerrain=None, doFeat=True, doBundle=True, doDense=True, pointmask=True, cleanpoints=True, fmethod=None, shpmask=None, subset=None, rep_dsm='0', egal=1, DegRap='0', slantr=False)¶

A function for the complete structure from motion process using the micasense red edge camera or Slant view camera.

The RGB imagery is used to generate DSMs, which are in turn used to orthorectify the remaining bands (Red edge, Nir).

Obviously the Malt and PIMs algorithms will perform better/worse than each other on certain datasets.

Notes

This assumes you have already generated a working directory with RGB and RRENir folders in it using the pycmac.mspec_proc function

This assumes certain parameters, if want fine-grained control, use the individual commands.

The absolute path needs to be provided for csv’s representing image data or calibration subsets

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
mode (string) – either Malt or PIMs
submode (string) – PIMs submode e.g. Forest, BigMac etc.
csv (string) – path to csv containing image xyz info in the micmac format
sub (string) – path to csv containing an image subset in the micmac format
ResolTerrain (string) – size of a the side of a pixel in the output dsm in metres (only with Malt!)
gpsAcc (string) – an estimate in metres of the onboard GPS accuracy
gcp (string) – whether to process gcps - you MUST have a GCP file in the MM format of #F=N X Y Z and MUST be in the working dir
gcpAcc (list (of strings)) – an estimate of the GCP measurment accuarcy [on the ground in metres, in pixels]
sep (string) – the csv delimiter if used (default “,”)
doFeat (bool) – if repeating after debug/changes mis out early stages
doBundle (bool) – if repeating after debug/changes mis out early stages
fmethod (bool) – feature search for image pairs strategy eg Line All
shpmask (string) – a shapefile mask to constrain malt-based processing
pointmask (bool) – use the micmac SaisieMasq3D (need QT compiled) to define a mask on the sparse cloud to constrain processing - this is VERY useful and will save a lot of time
subset (string) – a csv defining a subset of images to be processed during dense matching
rep_dsm (string) – sometimes it is necessary to redo the DSM (MEC folder) on the second 3-band composite run due to some sort of bug within MicMac, though most of the time this is not necessary def is ‘0’ change to ‘1’ if redoing dsm
cleanpoints (bool) – if true use the schnaps tie point cleaning tool for a evenly distributed

and reduced tie point set
slantr (bool) – if processing slantrange imagery change to true otherwise leave false for micasense

sfm.rgb_sfm(folder, proj='30 +north', ext='JPG', csv=None, sub=None, gpsAcc='1', gcp=None, gcpAcc=['0.03', '1'], sep=',', mode='PIMs', submode='Forest', ResolTerrain=None, doFeat=True, doBundle=True, doDense=True, fmethod=None, useGps=True, pointmask=True, shpmask=None, subset=None, egal=1, resize=None, cleanpoints=True)¶

A function for the complete structure from motion process using a RGB camera.

Obviously the Malt and PIMs algorithms will perform better/worse than each other on certain datasets.

Notes

This assumes certain parameters, if want fine-grained control, use the individual commands.

The absolute path needs to be provided for csv’s representing image data or calibration subsets

Parameters

folder (string) – working directory
proj (string) – a UTM zone eg “30 +north”
mode (string) – either Malt or PIMs
submode (string) – PIMs submode e.g. Forest, BigMac etc.
csv (string) – path to csv containing image xyz info in the micmac format
sub (string) – path to csv containing an image subset in the micmac format
ResolTerrain (string) – size of a the side of a pixel in the output dsm in metres (only with Malt!)
gpsAcc (string) – an estimate in metres of the onboard GPS accuracy
gcp (string) – whether to process gcps - you MUST have a GCP file in the MM format of #F=N X Y Z and MUST be in the working dir
gcpAcc (list (of strings)) – an estimate of the GCP measurment accuarcy [on the ground in metres, in pixels]
sep (string) – the csv delimiter if used (default “,”)
doFeat (bool) – if repeating after debug/changes mis out early stages
doBundle (bool) – if repeating after debug/changes mis out early stages
fmethod (bool) – select a feature detection strategy eg Line All etc
shpmask (string) – a shapefile mask to constrain malt-based processing
pointmask (bool) – use the micmac SaisieMasq3D (need QT compiled) to define a mask on the sparse cloud to constrain processing - this is VERY useful and will save a lot of time and is recommended over shpmask
subset (string) – a csv defining a subset of images to be processed during dense matching
cleanpoints (bool) – if true use the schnaps tie point cleaning tool for a evenly distributed and reduced tie point set