Unverified Commit 4c33e0a8 authored by Kris J. Becker's avatar Kris J. Becker Committed by GitHub
Browse files

Fix findfeatures matrix inversion issues and improve FastGeom performance (#4772) 

* findfeatures bug fixes and improvements

* Fix matrix inversion error on empty matrix. (Fixes #4639)
* Identify images that fail FastGeom transform and exclude from matching.
* Add TONOGEOM parameter to write failed file loads or FastGeom error file list if the transform cannot be determined
* Improve FastGeom transform algorithm using new radial point mapping scheme
* Add more debugging output to help diagnose problem images/procedures

* findfeatures - add Grid algorithm to FastGeom

* findfeatures mod to add Grid algorithm to FastGeom class
* Improved Grid algorithm by computing proper starting iteration to statisfy fastgeompoints request
* Added additional parameterization of Grid algorithm
* Reorganized mapping process to consolidate Radial and Grid algorithms in FastGeom

* findfeatures modifications after Astro code review

findfeatures modifications:

* Refactored FastGeom separating large sections of the compute() methods
* Redesigned the the radial algorithm for easier user configuration
* Added sanity checks avoid bad geometric correspondences
* Added GLOBALS parameters for easy configuration of FastGeom algorithms at runtime

* Fixed existing findfeatures test - error text

The ThreeImageNetwork.FunctionalTestFindfeaturesErrorNoInput expects an error to be thrown with spectific text. The text in findfeatures.cpp was changed in this PR. It fixes the existing findfeatures test suite.

* FastGeom.cpp updated to better accomodate testing

* Add findfeatures Radial/Grid config files for test

* Updated findfeatures test for Grid/Radial algos

* Fixed cnetwinnow test that created misplaced files

The cnetwinnow test needs tempDir().path() prepended to the “file_prefix” parameter to properly place the output files in the teardown directory.

* Small adjustment to new findfeatures tests

* Fixes/improvements to findfeature code

- Improved reporting of parameterizations of findfeatures
- Prevent creation of empty TONOTMATCH file when none are detected

* Significant modifications/improvements to docs

 - Added two new examples demonstrating/documenting the use of FASTGEOM algorithm, parameterization using GLOBALS and how to produce a regional mosaic using findfeatures with batch scripts.
- Reviewed, clarified and improved findfeatures program documentation

* Updated the change log

* Updates to findfeatures PR #4772

- Modified findfeatures.xml documentation to address PR review feedback
- Fixed use of projected images which wasn’t working due to improper instantiation of the cube projection object
- Updated CHANGELOG.md to better categorize all changes in this PR

* Removed scripts in example 4 of findfeatures docs

- Per request via USGS review, removed the Bash shell scripts that produce the results of example 4

- Removed the $ISISROOT/appdata/templates/findfeatures/mosaics containing the scripts

- Updated documentation in findfeatures.xml accordingly

- Updated CHANGELOG.md accordingly
parent af626634

CHANGELOG.md

+13 −0
Original line number Diff line number Diff line
@@ -35,11 +35,20 @@ release. 


## [Unreleased]





### Changed

- Removed the `.py` extention from the _isisdataeval_ tool `isisdata_mockup` for consistency and install it in $ISISROOT/bin; added the `--tojson` and `--hasher` option to _isisdata_mockup_ tool improve utility; updated the tool `README.md` documentation to reflect this change, removed help output and trimmed example results;  fixed paths to test data in `make_isisdata_mockup.sh`. [#5163](https://github.com/DOI-USGS/ISIS3/pull/5163)

- Significantly refactored FASTGEOM processing in <i>findfeatures</i> to accommodate stability and functionality. The scope of the algorithm was taken out of the ImageSource class and isolated to support this feature. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Report better information regarding the behavior of <i>findfeatures</i>, FASTGEOM algorithms, and creation of the output network. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)



### Added

- Added rclone to run dependencies in meta.yaml [#5183](https://github.com/DOI-USGS/ISIS3/issues/5183)

- Add new program option <b>TONOGEOM</b> to <i>findfeatures</i> that logs captures geometry errors in the FASTGEOM algorithm and records them to the file provided in this parameter. These images are excluded from the matching process. References [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Added a new Radial FASTGEOM transform mapping algorithm to <i>findfeatures</i> to address performance problems with the Grid algorithm. This is now the default algorithm if none are selected by the user (see the new <b>GLOBALS</b> parameter to specify the algorithm) [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Added new parameter <b>GLOBALS</b> to make parameterization of <i>findfeatures</i> behavior significantly easier and convenient. Wrote significant documentation for the parameter and provide several examples showing its use. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Added two new examples demonstrating/documenting the use of FASTGEOM algorithm, parameterization using <b>GLOBALS</b> and creation of a regional mosaic using <i>findfeatures</i>. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Added new option <b>GEOMSOURCE=BOTH</b> to <i>findfeatures</i> to check both the MATCH and FROM/FROMLIST images for valid control measure geometry to produce better networks and prevent downstream processing errors. Ignore points that end up with no valid measures (but can be retained with use of <b>PreserveIgnoredControl</b> via GLOBALS parameterization). [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Added new gtests for <i>findfeatures</i> that replaces all the old application tests. These tests are <i>FunctionalTestFindfeaturesFastGeomDefault</i>, <i>FunctionalTestFindfeaturesFastGeomRadialConfig</i>, <i>FunctionalTestFindfeaturesFastGeomGridDefault</i> and <i>FunctionalTestFindfeaturesFastGeomGridConfig</i>. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)



### Deprecated
@@ -49,6 +58,10 @@ release. 
- Updated History constructor to check for invalid BLOB before copying History BLOB to output cube [#4966](https://github.com/DOI-USGS/ISIS3/issues/4966)

- Updated photomet MinnaertEmpirical model to support photemplate-style PVL format [#3621](https://github.com/DOI-USGS/ISIS3/issues/3621)

- Fixed gaussstretch segmentation fault error and refactored gaussstretch files/tests to use gtest [#5240](https://github.com/DOI-USGS/ISIS3/issues/5240)

- Fix matrix inversion errors in <i>findfeatures</i> due to bad FASTGEOM matrix transforms using a more robust implementation to detect these errors and throw exceptions. Images with these errors are captured and logged to the <b>TONOTMATCHED</b> file. Fixes [#4639](https://github.com/DOI-USGS/ISIS3/issues/4639)

- Fixed <i>findfeatures</i> use of projected mosaics with correct check for <b>TargetName</b> in the Mapping labels. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)

- Fixed a bug in the <i>cnetwinnow</i> test that did not clean/remove it during test runs.

- Fixed <i>findfeatures</i> instantiation and use of projection classes to correctly return geometry data from projected images and mosaics. [#4772](https://github.com/DOI-USGS/ISIS3/issues/4772)



## [8.0.0] - 2023-04-19

isis/appdata/templates/findfeatures/findfeatures_fastgeom_defaults.pvl

0 → 100644
+40 −0
Original line number Diff line number Diff line 
Group = FastGeomCommonParameters

  # Defaults used for findfeatures FastGeom Algorithms



  # FastGeomAlgorithm           = radial

  FastGeomPoints               = 25

  FastGeomTolerance            = 3

  # GeomType                    = camera



  FastGeomQuerySampleTolerance = 0

  FastGeomQueryLineTolerance   = 0

  FastGeomTrainSampleTolerance = 0

  FastGeomTrainLineTolerance   = 0



  FastGeomDumpMapping = false

EndGroup = FastGeomCommonParameters





Group = FastGeomGridParameters

  # FastGeomAlgorithm            =  grid



  FastGeomGridStartIteration   = 0

  # FastGeomGridStopIteration    = 1 to N

  FastGeomGridIterationStep    = 1



  FastGeomTotalGridIterations  = 1



  FastGeomGridSaveAllPoints    = false

EndGroup = FastGeomGridParameters





Group = FastGeomRadialParameters

  # FastGeomAlgorithm           = radial



  FastGeomRadialSegmentLength  = 25

  FastGeomRadialPointCount     = 5

  FastGeomRadialPointFactor    = 1.0



  # FastGeomRadialSegments       = N

EndGroup = FastGeomRadialParameters

End

isis/src/control/apps/findfeatures/FastGeom.cpp

+647 −31

File changed.

Preview size limit exceeded, changes collapsed.

isis/src/control/apps/findfeatures/FastGeom.h

+47 −2
Original line number Diff line number Diff line
@@ -19,6 +19,7 @@ find files of those names at the top level of this repository. **/ 
#include "GenericTransform.h"

#include "MatchImage.h"

#include "PvlFlatMap.h"

#include "QDebugLogger.h"



namespace Isis {
@@ -52,19 +53,62 @@ class ImageTransform; 
 * @internal

 *   @history 2015-10-01 Kris Becker - Original Version

 *   @history 2016-04-06 Kris Becker Created .cpp file and completed documentation

 *   @history 2021-10-29 Kris Becker Added const qualifier to all compute() and

 *                                     apply() methods

 *   @history 2022-02-02 Kris Becker Added Grid implementation resulting in

 *                                     consolidation/abstraction of Radial

 *                                     algorithm code

 *   @history 2022-02-07 Kris Becker Modifications in response to USGS/Astro

 *                                     code review in PR #4772

 */



class FastGeom {

  public:

    typedef GenericTransform::RectArea   RectArea;

    typedef cv::Point2d                  FGPoint;

    typedef cv::Rect2d                   FGFov;



    FastGeom();

    FastGeom(const PvlFlatMap &parameters);

    FastGeom(const int maxpts, const double tolerance, const bool crop = false,

             const bool preserve = false, const double &maxarea = 3.0);

    virtual ~FastGeom();



    ImageTransform *compute(MatchImage &query, MatchImage &train);

    void apply(MatchImage &query, MatchImage &train);

    // ImageTransform *compute(MatchImage &query, MatchImage &train) const;

    ImageTransform *compute(MatchImage &query, MatchImage &train,

                            QLogger logger = QLogger() ) const;



    int radial_algorithm(MatchImage &query, MatchImage &train,

                        const FGFov &q_fov, const FGFov &t_fov,

                        const PvlFlatMap &parameters,

                        std::vector<FGPoint> &q_infov_points,

                        std::vector<FGPoint> &t_infov_points,

                        QLogger logger = QLogger()) const;



    int  grid_algorithm(MatchImage &query, MatchImage &train,

                        const FGFov &q_fov, const FGFov &t_fov,

                        const PvlFlatMap &parameters,

                        std::vector<FGPoint> &q_infov_points,

                        std::vector<FGPoint> &t_infov_points,

                        QLogger logger = QLogger()) const;



    void apply(MatchImage &query, MatchImage &train, QLogger logger = QLogger() ) const;



    static cv::Mat getTransformMatrix(const std::vector<FGPoint> &querypts,

                                      const std::vector<FGPoint> &trainpts,

                                      std::vector<uchar>         &inliers,

                                      const double tolerance,

                                      QLogger logger = QLogger() );



    PvlFlatMap getParameters() const;



    void dump_point_mapping(MatchImage &query, MatchImage &train,

                          const QString &method, const PvlFlatMap &parameters,

                          const std::vector<FGPoint> &q_points,

                          const std::vector<FGPoint> &t_points,

                          const std::vector<SurfacePoint> &q_surface_points,

                          const std::vector<bool> &t_inFOV,

                          QLogger logger = QLogger()) const;



  private:

    int        m_fastpts;    //!< Number of points to use for geom
@@ -75,6 +119,7 @@ class FastGeom { 
    PvlFlatMap m_parameters; //!< Parameters of transform



    void validate( const QString &geomtype ) const;



};



}  // namespace Isis

isis/src/control/apps/findfeatures/FeatureAlgorithmFactory.cpp

+25 −1
Original line number Diff line number Diff line
@@ -206,6 +206,30 @@ const PvlFlatMap &FeatureAlgorithmFactory::globalParameters() const { 
}





/**

 * @brief Parse global parameters into a Pvl flat map strucure

 *

 * This method will accept a string that contain variables structured according

 * to specifications used in the algorithm string. It only recognizes

 * keyword/variable structures of the form "keyword1:value1@keyword2:value2".

 *

 * @param globals String of global parameters

 */

PvlFlatMap FeatureAlgorithmFactory::parseGlobalParameters(const QString &globals) {

  PvlFlatMap pvlmap;

  QStringList parms = globals.split("@", QString::SkipEmptyParts);

  for (int i = 0 ; i < parms.size() ; i++ ) {



    // Only parse substrings that have 2 distinct parts separated by :

    QStringList parts = parms[i].split(":", QString::SkipEmptyParts);

    if ( parts.size() == 2 ) {

      pvlmap.add(parts[0], parts[1]);

    }

  }



  return ( pvlmap );

}



/**

 * @brief Set the global parameters to use in all matchers created

 *
@@ -391,7 +415,7 @@ RobustMatcherList FeatureAlgorithmFactory::create(const QString &specs, 
 *

 *  In addition, parameters that alter the behavior of the outlier detection

 *  processing, among oother things, in the RobustMatcher can be specified as an

 *  additional part of the string using the "/paramters@name:value..."

 *  additional part of the string using the "/parameters@name:value..."

 *  specification.

 *

 * @param definition A single string specification for an OpencCV feature-based