Figure 5. (a) Drawing of a dog, note that the drawing is relatively coarse and the lines are relatively broad. (b) The 2D drawing of the dog is mapped to the TP using the nearest points on the TP method.
The accuracy of the mapping from 2D to 3D is dependent on the geometry of the large-scale structure and the position of the drone relative to the structure. If the image of the camera sensor is tangential to the tower (the red surfaces in the figures above) then the mapping will be very close to perfect when the local surface of the large-scale structure can be approximated by a plane surface. The local curvature of a plane surface is infinite, decreasing the large structure’s local curvature will decrease the mapping accuracy because the 2D image pixels will get compressed when they are mapped onto the tower. In the case of the transition pieces most of the surfaces of interest will have a very large local radius given a high mapping accuracy. Using the correct values of the digital camera’s focal length and sensor dimensions in the equations above ensures that the sizes and distances in the images will get mapped correctly to the large-scale structure.
Demonstration of mapping surface paint damage on transition piece
Using the techniques discussed in the previous section it is possible to map all the paint damages found in images acquired during a drone flight. The paint damage pixels are found using the YOLO and color threshold algorithms. The mapping of the paint damage pixels is here performed using the projection along surface normals method. The results are presented in Figure 6 (a) below. A CAD model of the TP moved to the geo-referenced coordinate system is used here. It is seen that the majority of the paint damages are concentrated on the lower section of the TP. This information can be used in the optimization of TP production. The mapped paint damages are divided into clusters in Figure 6 (b). This is done in order to get an overview of the areas where the damages are located. If the smallest distance between paint damages in different clusters is chosen to be 75 cm the total number of clusters becomes seven. The points in each of the clusters have a unique color. The surface areas for the clusters are seen in the figure together with image numbers. This makes it possible to locate the images that have paint damage in a specific cluster. The largest cluster has paint damage that can be seen in 8 different images. It is seen that the paint damage surface areas for most of the clusters are as expected small. Cluster 1 has however a large surface area because many small damages are scattered around this area. The paint damages in Figure 2 (a) to (d) will all be mapped to this cluster. The unit for the surface areas is a meter. Figure 6 (c) shows the 3D reconstructed model based on all the images captured during the flight. The photogrammetry software ContextCapture from Bentley Institute, see [24] has been used to generate the 3D reconstruction model. The quality of the 3D geometric reconstruction can be improved due to a limited number of images, i.e., only 445 images were available in this inspection. The number of tie-points calculated and used in ContextCapture is relatively low on the cylindrical part of the TP resulting in the poor 3D reconstruction of the cylindrical part of the TP. The low number of tie-points is caused by very little texture in these sections of the images, large areas with only slight color differences can be seen in the images. The number of tie points on the TP platforms is higher leading to a better 3D reconstruction. The blue points in Figure 6 (c) show the mapped paint damages. When the Digital Twin is applied in an operational environment the paint damages from the images will normally be mapped to a 3D CAD model of the TP. The reason for this is the long time it takes to calculate a 3D reconstruction of the TP with the use of photogrammetry, more than 8 hours on an HP workstation, and the images needed to detect paint damage are often not the same type of images that can be used in the 3D reconstruction. These images must have the content of the TP but also some parts of the surrounding areas. The total number of needed images becomes very large and the acquisition therefore a time-consuming process.