Test plan

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


here are some thoughts about a static test framework for gegl operations.

* Functional Tests
These can be as simple as a set of files including a composition,
reference output and a test description. A perl/ruby script feeds the
composition through the gegl binary and compares the output with
reference data. Optionally some PNGs + HTML are created for visual inspection.

Test Case:
  foo.xml                     composition                     reference data; format to be determinded
  foo.specs.yaml              test specification              optional input data

  output/              compositing result
  output/foo.result.yaml      pass/fail info + basic statistic measures

Optional report output:

The test specification foo.specs.yaml looks like:

  comment: Regression test for Bug #1234
      peak-absolute:   1e-6

The result foo.result.yaml something like:

  testname : foo
  result   : pass
  errors   :
      peak-absolute    : 1e-10
      mean-square      : 1e-15
      differing-pixels : 123
  timing  :
      total     : 500       # ms

The data format .xxx should be a float format understood by ImageMagick,
candidates so far are MIFF, PFM, FITS, TIFF.

Reference data is most easily created using Octave. Probably a 'make reference' target
will generate them from some ground truth m-files.

* Unit Tests
Unit tests for operations are most comfortably expressed as functional tests.
However, there are some tests which are common for all/many operations:
- rendering must be independent of tile size
- operations with spatial parameters should be scale invariance, i.e.
  applying a operation on a downscaled composition using parameters which have been
  downscaled by the same factor should give a good approximation of applying the
  unscaled operation on the unscaled composition and downscaling afterwards.
- neutral settings

Somewhat unrelated, i'd also like to install a round trip test for GEGL's
XML format, but haven't found a good way to deal with value rounding yet.

Comments, other things to consider, do's and don'ts?
Is somebody already working on this?

Gegl-developer mailing list

[Video For Linux]     [Photo]     [Yosemite News]    [Yosemite Photos]    [gtk]     [GIMP Users]     [KDE]     [Scanner]     [Gimp's Home]     [Gimp on Windows]     [Steve's Art]     [Webcams]