Figure 5. (a) Drawing of a dog, note that the drawing is
relatively coarse and the lines are relatively broad. (b) The 2D drawing
of the dog is mapped to the TP using the nearest points on the TP
method.
The accuracy of the mapping from 2D to 3D is dependent on the geometry
of the large-scale structure and the position of the drone relative to
the structure. If the image of the camera sensor is tangential to the
tower (the red surfaces in the figures above) then the mapping will be
very close to perfect when the local surface of the large-scale
structure can be approximated by a plane surface. The local curvature of
a plane surface is infinite, decreasing the large structure’s local
curvature will decrease the mapping accuracy because the 2D image pixels
will get compressed when they are mapped onto the tower. In the case of
the transition pieces most of the surfaces of interest will have a very
large local radius given a high mapping accuracy. Using the correct
values of the digital camera’s focal length and sensor dimensions in the
equations above ensures that the sizes and distances in the images will
get mapped correctly to the large-scale structure.
Demonstration of mapping surface paint damage on transition
piece
Using the techniques discussed in the previous section it is possible to
map all the paint damages found in images acquired during a drone
flight. The paint damage pixels are found using the YOLO and color
threshold algorithms. The mapping of the paint damage pixels is here
performed using the projection along surface normals method. The results
are presented in Figure 6 (a) below. A CAD model of the TP moved to the
geo-referenced coordinate system is used here. It is seen that the
majority of the paint damages are concentrated on the lower section of
the TP. This information can be used in the optimization of TP
production. The mapped paint damages are divided into clusters in Figure
6 (b). This is done in order to get an overview of the areas where the
damages are located. If the smallest distance between paint damages in
different clusters is chosen to be 75 cm the total number of clusters
becomes seven. The points in each of the clusters have a unique color.
The surface areas for the clusters are seen in the figure together with
image numbers. This makes it possible to locate the images that have
paint damage in a specific cluster. The largest cluster has paint damage
that can be seen in 8 different images. It is seen that the paint damage
surface areas for most of the clusters are as expected small. Cluster 1
has however a large surface area because many small damages are
scattered around this area. The paint damages in Figure 2 (a) to (d)
will all be mapped to this cluster. The unit for the surface areas is a
meter. Figure 6 (c) shows the 3D reconstructed model based on all the
images captured during the flight. The photogrammetry software
ContextCapture from Bentley Institute, see [24] has been used to
generate the 3D reconstruction model. The quality of the 3D geometric
reconstruction can be improved due to a limited number of images, i.e.,
only 445 images were available in this inspection. The number of
tie-points calculated and used in ContextCapture is relatively low on
the cylindrical part of the TP resulting in the poor 3D reconstruction
of the cylindrical part of the TP. The low number of tie-points is
caused by very little texture in these sections of the images, large
areas with only slight color differences can be seen in the images. The
number of tie points on the TP platforms is higher leading to a better
3D reconstruction. The blue points in Figure 6 (c) show the mapped paint
damages. When the Digital Twin is applied in an operational environment
the paint damages from the images will normally be mapped to a 3D CAD
model of the TP. The reason for this is the long time it takes to
calculate a 3D reconstruction of the TP with the use of photogrammetry,
more than 8 hours on an HP workstation, and the images needed to detect
paint damage are often not the same type of images that can be used in
the 3D reconstruction. These images must have the content of the TP but
also some parts of the surrounding areas. The total number of needed
images becomes very large and the acquisition therefore a time-consuming
process.