Example a/optimisation/host_mesh_fit: Host mesh fit : Fitting a face to a fox

This example shows how to find embedded locations of data points and nodes in a host mesh, how to move the host mesh both interactively and by fitting the host mesh to target data point coordinates, with smoothing, and how to update the coordinates of embedded nodes in the fitted host.
Initial host mesh and human face. Fitted host mesh and fox face showing error projections.

Screenshot of example a/optimisation/host_mesh_fit


The comfile run by this example is as follows:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# ----------
#
# Cmgui example: Host Mesh Fit
#
# Use host mesh fitting to fit a face to a fox.
#
# This example shows how to find embedded locations of data points and nodes
# in a host mesh, how to move the host mesh both interactively and by fitting
# the host mesh to target data point coordinates, with smoothing, and how to
# update the coordinates of embedded nodes in the fitted host.
#
# This example uses new features of Cmgui to achieve the same result as example
# ap (which used old back-end CMISS-cm to find xi and perform the fit).
#
# ----------
#
gfx define tessellation default minimum_divisions "1" refinement_factors "8";
#
gfx create material foxy normal_mode ambient 0.61 0.29 0 diffuse 0.64 0.26 0 emission 0 0 0 specular 0.5 0.5 0.4 alpha 1 shininess 0.6;
gfx create material purple ambient 0.72 0 1 diffuse 0.5 0 1 emission 0 0 0 specular 1 1 1 alpha 1 shininess 0.8;
#
# Read in the host mesh in which we will embed our face mesh.
# It consists of four cube elements large enough to fully enclose our mesh,
# and uses bicubic-linear interpolation for the 'host_coordinates' field
# to give the desired smooth fit in the plane of the face.
gfx read nodes $example/host.exnode;
gfx read elements $example/host.exelem;
#
# Rename field host_coordinates to keep reference value for smoothing:
gfx modify field host_coordinates rename host_reference_coordinates;
# Re-read field host_coordinates which is to be fitted:
gfx read nodes $example/host.exnode;
gfx read elements $example/host.exelem;
#
# Read in the embedded face mesh in its original position.
gfx read nodes $example/face.exnode
gfx read elements $example/face.exelem
#
# Visualise the host mesh and the face:
gfx modify g_element "/" lines subgroup host coordinate host_coordinates material default;
gfx modify g_element "/" surfaces domain_mesh2d subgroup host coordinate host_coordinates tessellation default LOCAL select_on material blue selected_material default_selected render_wireframe;
gfx modify g_element "/" lines subgroup face coordinate coordinates material default;
gfx modify g_element "/" surfaces subgroup face coordinate coordinates material foxy;
gfx modify g_element "/" node_points subgroup face coordinate coordinates glyph sphere size "5*5*5" no_select material purple;
if (!$TESTING)
{
	gfx create window 1;
	gfx modify window 1 view perspective eye_point 360.515 -98.2134 593.377 interest_point 27.6929 18.9214 48.7295 up_vector 0.0456354 0.981996 0.183306 view_angle 41.4883 near_clipping_plane 6.48947 far_clipping_plane 2319.12;
  gfx modify window 1 layout width 700 height 700;
}
#
# Create a field which dynamically finds the location on the 3-D host mesh at
# which its host_reference_coordinates equal the value of the coordinates field.
# [Note: In Cmgui 2.9 rename mesh3d -> cmiss_mesh_3d]
gfx define field find_host_location find_mesh_location find_exact mesh host.mesh3d mesh_field host_reference_coordinates source_field coordinates;
#
# Define a field for storing these locations so we don't need to
# expensively recalculate them:
gfx define field stored_host_location finite_element element_xi;
# Allocate storage for the stored_location field on nodes of the face:
gfx modify nodes group face define stored_host_location;
# Assign stored_host_location field from find_host_location at face nodes:
gfx evaluate ngroup face destination stored_host_location source find_host_location;
#
# Define a field which gives the host_coordinates at the stored_host_location
gfx define field stored_host_coordinates embedded element_xi stored_host_location field host_coordinates;
#
# Visualise vectors on the face from coordinates to stored_host_coordinates:
gfx define field delta_coordinates add fields stored_host_coordinates coordinates scale_factors 1 -1;
gfx modify g_element "/" points as green_lines domain_nodes coordinate coordinates tessellation default_points LOCAL glyph line size "1*1*1" offset 0,0,0 font default orientation delta_coordinates scale_factors "1*1*1" select_on material green selected_material default_selected render_shaded;
#
# Visualise host mesh nodes and (when selected) derivatives so they can be edited interactively:
gfx define field dx_ds1 node_value fe_field host_coordinates d/ds1;
gfx define field dx_ds2 node_value fe_field host_coordinates d/ds2;
gfx modify g_element "/" points domain_nodes subgroup host coordinate host_coordinates tessellation default_points LOCAL glyph sphere size "10*10*10" offset 0,0,0 font default select_on material default selected_material default_selected render_shaded;
gfx modify g_element "/" points domain_nodes subgroup host coordinate host_coordinates tessellation default_points LOCAL glyph arrow_solid size "0*10*10" offset 0,0,0 font default orientation dx_ds1 scale_factors "1*0*0" draw_selected material silver selected_material silver render_shaded;
gfx modify g_element "/" points domain_nodes subgroup host coordinate host_coordinates tessellation default_points LOCAL glyph arrow_solid size "0*10*10" offset 0,0,0 font default orientation dx_ds2 scale_factors "1*0*0" draw_selected material gold selected_material gold render_shaded;
#
# At this point you can switch to the 'node tool' on the graphics window, enable edit
# and click on nodes of the host mesh to edit their positions and derivatives.
# When you try this you will see that the green lines from the face nodes to their
# embedded locations in the host mesh grow.
# Unfortunately, we can't interpolate embedded coordinates directly, so we need to
# evaluate the embedded coordinates into the coordinates of the face periodically with:
gfx evaluate ngroup face source stored_host_coordinates destination coordinates;
# The 'node tool' in the Cmgui graphics window has a nifty feature enabling a
# command to be called after each node is edited: define a string_constant field
# containing the above command and set it as the command_field in the node tool:
if (!$TESTING)
{
  gfx define field update_face_command string_constant "gfx evaluate ngroup face source stored_host_coordinates destination coordinates;"
  gfx modify window 1 node_tool command_field update_face_command edit;
}
# Now you can select and drag host mesh nodes and derivatives, and the face will
# update automatically when you release the mouse. Play around with this and
# see the limits of what the host mesh can morph the face into. If it isn't able
# to give the changes you want, you may need a better host mesh.
#
# Now reset the host and face coordinates for the second half of this example: fitting.
gfx read nodes $example/host.exnode;
gfx evaluate ngroup face source stored_host_coordinates destination coordinates;
gfx modify g_element "/" points as green_lines invisible;
#
# To perform a host mesh fit, we need an objective function that is to be
# minimised. Here we will read in a set of datapoints with initial and
# target coordinates; the initial coordinates say where the datapoints are
# relative to the host_reference_coordinates; the target coordinates say
# where we want those locations to move to in the fit.
#
# In this made-up example, our data points coincide with the initial face
# coordinates, and the target coordinates are positions with a fox-like layout.
# In a real problem the data points need not match any of the coordinates of the
# embedded mesh: they may be a subset of them (e.g. end of the nose, edges of the
# mouth, eyes etc.) or arbitrary positions on the embedded mesh or in the host.
# The important point is that they are merely data for fitting updated coordinates
# of the host mesh. It is the mapping of all coordinates from the initial (reference)
# coordinates of the host to the final coordinates that enables us to transform
# any other embedded points, including in this case the face mesh.
#
# Read in a set of data points with 'source_coordinates' and 'target_coordinates':
gfx read data $example/datapoints_source.exdata;
gfx read data $example/datapoints_target.exdata;
# 
# Find and store the locations in the host mesh of the source_coordinates. Note that
# Cmgui allows datapoints and nodes/elements to have the same field defined on them,
# so we could have made this simpler by reading in source_coordinates as 'coordinates',
# and re-using the definition of the find_host_coordinates field. But we didn't:
gfx define field find_host_location_source find_mesh_location find_exact mesh host.mesh3d mesh_field host_reference_coordinates source_field source_coordinates;
# However, we will reuse the 'stored_host_location' field, since that enables
# all the embedded fields to be used unchanged:
gfx modify data group data define stored_host_location;
gfx evaluate dgroup data destination stored_host_location source find_host_location_source;
#
# Now visualise the error between the target_coordinates and the current
# embedded locations in the host mesh:
gfx define field error add fields target_coordinates stored_host_coordinates scale_factors 1 -1;
gfx define field mag_error magnitude field error;
gfx modify g_element "/" points domain_datapoints subgroup data coordinate stored_host_coordinates tessellation default_points LOCAL glyph line size "0*1.8*1.8" offset 0,0,0 font default orientation error scale_factors "1*0*0" no_select material red data mag_error spectrum default selected_material default_selected render_shaded;
gfx modify spectrum default autorange;
gfx modify spectrum default minimum 0.0;
#
# Next form the sum of squares of the error, which we want to minimise:
gfx define field data_objective nodeset_sum_squares field error nodeset data.datapoints;
#
# Fitting is not possible with just the above data_objective, because the number
# of points is insufficient to get a solution. In particular, the data points are
# essentially two-dimensional in the plane of the face so the 'xi3' direction
# of the host mesh is effectively unconstrained.
# Hence it is necessary to add a volume smoothing objective, which penalises strains
# or curvatures of the host mesh with a modest weighting to not overly
# affect fitting of the data points.
#
# First make a penalty consisting of scaled displacement gradient:
gfx define field host_displacement add fields host_coordinates host_reference_coordinates scale_factors 1 -1;
gfx define field host_displacement_gradient gradient field host_displacement coordinate host_reference_coordinates;
gfx define field alpha constant 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01;
gfx define field scaled_host_displacement_gradient multiply_components fields host_displacement_gradient alpha;
#
# Next compute a penalty consisting of scaled curvatures i.e. 2nd derivatives w.r.t. xi.
# Note that curvatures can currently only be computed for 'finite element' fields
# i.e. those directly interpolated over elements and read in from EX files.
# We won't penalise curvature in the xi3 direction since interpolation is linear in xi3.
gfx define field host_curvature11 basis_derivative fe_field host_coordinates order 2 xi 1 1;
gfx define field host_curvature22 basis_derivative fe_field host_coordinates order 2 xi 2 2;
gfx define field host_curvature composite host_curvature11 host_curvature22;
gfx define field beta constant 0.00003 0.00003 0.00003 0.00003 0.00003 0.00003;
gfx define field scaled_host_curvature multiply_components fields host_curvature beta;
#
# Now form the integral of squares of the concatenation of these penalties:
gfx define field total_smoothing composite scaled_host_displacement_gradient scaled_host_curvature;
gfx define field volume_smoothing_objective mesh_integral_squares integrand_field total_smoothing coordinate host_reference_coordinates mesh host.mesh3d gaussian_quadrature numbers_of_points "4*4*2";
#
# Perform the minimisation, optimising host mesh node coordinates:
gfx minimise objective_fields data_objective & volume_smoothing_objective independent_fields host_coordinates conditional_fields host LEAST_SQUARES_QUASI_NEWTON maximum_iterations 10 hide_output;
# Update the face to the fitted host mesh:
gfx evaluate ngroup face source stored_host_coordinates destination coordinates;
# You can see how difficult it was to fit around the snout; to do a better
# job would require more host mesh elements in this area.
#
if ($TESTING) {
  gfx write nodes group host field host_coordinates fitted_host.exnode;
}
#
# Note that the above alpha and beta constants have been found by trial and error.
# In your fitting, vary the values by whole magnitudes or more then fine tune
# until you get a result that is both a good fit for the datapoints, and yet
# smooth enough to give reasonable deformations away from the datapoints.
#
# So what have we achieved here? Wasn't that a very complicated way to move the nodes?
# True but now we have a mapping from the first face to the second so
# can update a high res version of the first face to approximate the second,
# or if we had a standard human body model we can fit some important markers
# measured on a specific individual, fit to those and then update the whole
# model to approximate that individual.
#

Files used by this example are:

Name                      Modified     Size

host_mesh_fit.com 04-Aug-2014 12k COPYRIGHT 04-Aug-2014 504 datapoints_source.exdata 04-Aug-2014 4.1k datapoints_target.exdata 04-Aug-2014 4.2k face.exelem 04-Aug-2014 18k face.exnode 04-Aug-2014 4.1k host.exelem 04-Aug-2014 7.4k host.exnode 04-Aug-2014 5.4k

Download the entire example:

Name                                          Modified     Size

examples_a_optimisation_host_mesh_fit.tar.gz 09-Mar-2016 338k

Testing status by version:

StatusTestedReal time (s)
i686-linux
cmgui-wxFailureSun Mar 6 00:05:44 20161
last breakMon Aug 25 13:12:00 20142
cmgui-wx-debugFailureSun Mar 6 00:05:44 20161
last breakMon Aug 25 13:12:00 20142
cmgui-wx-debug-memorycheckFailureSun Mar 6 00:05:45 20161
last breakMon Aug 25 13:12:00 20142
cmgui-wx-debug-valgrindFailureSun Mar 6 00:38:27 201624
last breakTue Feb 24 00:07:00 201523
x86_64-linux
cmgui-wxFailureSun Mar 6 00:01:31 20160
last breakFri Aug 15 00:33:00 20141
cmgui-wx-debugFailureSun Mar 6 00:01:31 20160
last breakFri Aug 15 00:33:00 20140
cmgui-wx-debug-memorycheckFailureSun Mar 6 00:01:32 20161
last breakFri Aug 15 00:33:00 20141
cmgui-wx-debug-valgrindFailureSun Mar 6 00:04:15 201619
last breakSun Mar 6 00:03:00 201619
cmgui-wx-gcc-cad-debug-valgrindSuccessThu Jan 7 00:05:11 201617

Testing status by file:


Html last generated: Wed Mar 9 16:02:26 2016

Input last modified: Wed Mar 9 15:49:41 2016


CMISS Help / Examples / a / optimisation / host_mesh_fit