Abstractions Engine

#include <abstractions/engine.h>
using namespace abstractions;

Expected<Image> abstractions::RenderImageAbstraction(const int width, const int height, const Options<render::AbstractionShape> shapes, ConstRowVectorRef solution, const double alpha_scale = 1.0, const Pixel background_colour = Pixel(0, 0, 0, 255))

Render an image abstraction with the provided configuration.

Parameters:

• width – output image width

• height – output image height

• shapes – shape configuration

• solution – solution vector

• alpha_scale – alpha scaling

• background_colour – (optional) background colour

Returns:

rendered image abstraction

class Engine

Given an image, generate an abstract representation using simple shapes.

The Engine uses the PgpeOptimizer to find the “optimal” combination of shapes to represent an image. The end result will always be slightly different because PGPE is a stochastic optimization algorithm; it uses random sample to move towards a local optima.

Public Functions

Expected<OptimizationResult> GenerateAbstraction(const Image &reference) const

Generate an abstract representation from the provided reference image.

Parameters:

reference – reference image

Returns:

the results of the optimization, or an error if the optimization failed

void SetCallback(const std::function<void(int, double, ConstRowVectorRef)> &cb)

Set the callback that runs after an optimization step.

Parameters:

cb – a callback function that takes the current iteration, total number of iterations, solution cost, and the current solution

Public Static Functions

static Expected<Engine> Create(const EngineConfig &config, const PgpeOptimizerSettings &optim_settings = PgpeOptimizerSettings())

Create a new abstractions engine.

Parameters:

• config – engine configuration

• optim_settings – optional optimizer configuration

Returns:

the initialized engine or an error if the configuraiton failed

struct EngineConfig

Engine configuration options.

Public Functions

Error Validate() const

Validate the Engine configuration.

Returns:

an error if the configuration was invalid

Public Members

int iterations = 10000

Total number of optimizer iterations.

int num_samples = 256

The number of samples to draw when calculating the reward costs.

The PGPE optimizer generates a set of random samples around the current “best” solution. It then expects the user to tell it how “good” the solutions are.

Options<render::AbstractionShape> shapes = render::AbstractionShape::Triangles

The set of shapes to use for the image abstraction.

The optimizer can support multiple shape types. The default is just triangles, but different ones may be used and even combined. Using multiple shapes will slow down the optimizer since it increases the length of the parameter vector.

See the AbstractionShape enum for the list of available shapes.

double alpha_scale = 1.0

A scaling factor applied onto the shapes’ alpha channel.

The default is ‘1.0’, meaning shapes can be completely opaque. Setting the value to something lower can “soften” the abstraction by forcing shapes to always translucent.

This value must be greater than 0 and less than or equal to 1.0.

int num_drawn_shapes = 50

The number of shapes, per shape type, to draw.

This, along with the shapes option, controls how “abstract” the final result will be. This number is the number of shapes that the engine is allowed to use when reconstructing the input image. The total number of shapes will always be this value times the number of shape types.

ImageComparison comparison_metric = ImageComparison::L2Norm

The image comparison metric used to calculate the matching cost.

The metric affects how fine details in the image are treated.

std::optional<int> num_workers = {}

The number of worker threads used during the optimization.

The default is to let the internal thread pool pick the number of workers.

std::optional<DefaultRngType::result_type> seed = {}

Set the base seed for the PRNGs used by the optimizer.

A randomly generated seed will be used if one isn’t specified. Otherwise the seed can be provided for some degree of repeatabilty.

Note

Each sample renderer has its own PRNG. Because this is the base seed, each PRNG obtains its seed from this one. Both seed and num_samples must be the same for a result to be repeatable.

enum class abstractions::ImageComparison

The type of image comparison the abstraction engine should use.

Values:

enumerator L1Norm

Compare images using the sum of absolute differences.

enumerator L2Norm

Compare images using the sum of squared differences.

struct OptimizationResult

The results of an optimization from the abstractions Engine.

Public Functions

Error Save(const std::filesystem::path &file) const

Save the optimization result to a file.

Parameters:

file – file name

Returns:

an Error if the result could not be saved

Public Members

RowVector solution

The solution that best represents the input image.

double cost

The final optimization cost.

int iterations

The number of iterations the optimization ran for.

double aspect_ratio

Aspect ratio (width over height) of the source image.

double alpha_scaling

Alpha scaling factor used during rendering.

Options<render::AbstractionShape> shapes

The shapes used in the reconstruction.

DefaultRngType::result_type seed

The PRNG seed used by the optimizer.

TimingReport timing

Details about how long each stage in the abstraction process took.

Public Static Functions

static Expected<OptimizationResult> Load(const std::filesystem::path &file)

Load an optimization result from a file.

Parameters:

file – file name

Returns:

the OptimizationResult or an error if it could not be loaded

struct TimingReport

The timing for each stage of the optimization pipeline.

Public Types

using Duration = std::chrono::microseconds

Alias for “microseconds”.

Public Functions

TimingReport(int iterations, int samples)

Create a timing report, allocating all necessary memory.

Parameters:

• iterations – number of iterations

• samples – number of samples used in the optimization loop

inline int NumIterations() const

Number of iterations.

inline int NumSamples() const

Number of samples during the render-and-compare step.

Public Members

Duration total_time

The total time the abstraction generation took.

Stages stages

The per-stage timing, as seen by the main thread.

Iterations iterations

The time spent during any single iteration.

struct Iterations

The time spent within a single iteration, directly measured by by the worker thread.

This is not directly comparable to the timings in the Stages struct because they are measured by a thread worker. They execute on separate threads and much of the work is done in parallel.

Public Members

std::vector<Duration> sample

The time spent, per iteration, generating candidate samples.

std::vector<Duration> render_and_compare

The time spent, per iteration, rendering and computing the cost for a generated sample.

The samples are stored as a <iterations> x <samples> array.

std::vector<Duration> optimize

The time spent, per iteration, on running the PGPE optimizer.

std::vector<Duration> callback

The time spent, per iteration, on invoking the user callback.

struct Stages

Information about the aggregate stage timings.

The stage times are the times as seen by the main thread. The work is is done on a separate thread pool so this includes any overhead that comes from waiting for tasks to complete.

Public Members

Duration initialization

The time needed to initialize the abstraction engine.

Duration sample

The total time spent on generating candidate samples.

Duration render_and_compare

The total time spent rendering and computing the cost for the generated samples.

Duration optimize

The total time spent running the optimizer.

Duration callback

The total time spent on invoking the user callback.