In-process inspection of lattice geometry with laser line scanning and optical tomography in fused filament fabrication

. One of the challenges of lattice manufacturing by fused filament fabrication is to achieve geometric accuracy of the internal reticular structures. In this work a solution for in-process inspection is presented, based on combining a custom laser scanner system, mounted into the fabrication machine, and a method for optical tomography. The scanner allows for 2.5D layer measurement, with superior detection of layer edges with respect to 2D optical imaging. Optical tomography is then achieved by vertical stacking of reconstructed layer boundaries, leading to a full volumetric reconstruction of the lattice as a voxel model. Inspection can be performed layer-wise, by comparing the current slice measured by the laser scanner with a reference virtual layer obtained by simulation of the deposition process, or on entire portions of reconstructed 3D geometry, by performing voxel-wise comparisons in 3D, to identify local missing or excess deposited material. The proposed solution proves capable of monitoring an evolving 3D part geometry, allowing also the observation of internal regions, invisible when using conventional optical, post-process imaging methods.


Introduction
Amongst additive manufacturing technologies, fused filament fabrication (FFF) is gaining traction beyond rapid prototyping, and is increasingly adopted to fabricate high-value added parts of significant geometric complexity [1].FFF is a material extrusion process based on thermal reaction bonding (MEX-TRB [2]) where the material (polymer of composite with polymer matrix) is provided in form of a filament.FFF allows the realization of hollow structures as well as trabecular, lattice and otherwise reticular patterns, through layer-based fabrication.However, as geometric requirements become increasingly more stringent, geometric inspection becomes more challenging, in particular as it is impossible to access the internal structures for measurement with contact or non-contact optical means.The only current approach to measure the internal regions of a hollow or reticular structure is via X-ray CT scanning [3][4][5].Apart from the issues of metrological uncertainty and the costs/complexity of the approach, measurement can only be executed post-process, thus any imperfection originated in the early stages of fabrication can be detected only after the part has been completed, leading to significant waste of time, energy and material in case of part rejection.A viable alternative is optical tomography, a method of inprocess measurement where 2D images of each layer are taken using optical cameras, typically observing from above [6][7][8][9][10].Image processing can be used to reconstruct the arrangement of material and void regions within each image.The advantage is that anomalies can be detected as they occur within each layer.Moreover, if required when the part is complete, images can be vertically stacked on top of each other as if they were slices of the part, thus allowing to obtain a full 3D dataset depicting external and internal structures [6].Optical tomography is however, itself characterized by significant, currently unsolved challenges.The reconstruction of the spatial arrangement of material and voids in each layer is based on image processing, which in turn is influenced by illumination, optical properties of the material and optical property of the camera [11].Self and projected shadows, focus issues, and inherent complexity of observing structures on a layer which are barely discernible from those of the layers underneath, make the identification of the internal and external layer contours very challenging.Recent work [6] has shown that contour identification based on edge detection can be improved by using a digital twin whose purpose is to narrow down the regions within the image that should be searched for edges.The method consists of using a simulation of the FFF extrusion process to generate a virtual copy of the image that a camera is expected to see if observing from above.As the virtual image is semantically labelled, the camera can use it to extract the regions where the real edge is supposedly located, and search for edges only within those regions [6].Objective of this work is to provide an alternative to the approach illustrated in [6] by exploring: -the use of laser line scanning to replace optical 2D imaging.The transition to 2.5D topography acquisition is designed to provide a more robust means to identify internal and external layer contours within each layer; -the use of digital geometric models replicating expected layer geometry, to allow for a direct comparison between expected and measured results for each layer, so that warnings can be immediately generated upon identification of discrepancies.The use of laser line scanning technologies to observe the part in FFF has been attempted before [12][13][14].The proposed method presents specific elements of distinction with respect to the state of the art: -there is no need to perform a registration process between the geometry measured by laser scanning and the reference geometry used for comparison.Our laser scanning solution is integrated within the FFF machine, uses the same axes, and relies on the same axis encoders.The two datasets exist within the same reference frame, and alignment features the same positioning accuracy achieved in fabrication; -our laser line scanning solution is not configured to perform a 2.5D measurement of the layer, but rather works as a robust, 2D detector of material boundaries, by recognising the points where material falls below a reference band representing the expected layer height.Whilst this approach does not let us investigate phenomena that require the reconstruction of the entire 2.5D map of layer heights (like in [14] for warpage), it provides an effective boundary detection functionality, superior to edge detection in 2D layer images, which can instead be influenced by shadows and focusing issues [6].

Materials and Methods
Hardware.A laser line scanner was built in-house, and mounted on a prototype FFF machine being developed at our labs (Fig. 1).The laser source (80mW@520nm) [15] is pointed vertically downwards onto the current layer and is observed by a tilted optical microscope (5 megapixels, 10× -220× magnification -Dinolite AM7915MZT [16]).The scanner is rigidly connected to the machine z-axis and placed so the when the laser line is projected on the layer surface, it falls exactly in the middle of the field of view and at focal range of the tilted optical microscope.The scanner is operated by a dedicated Arduino controller [17].The laser line is aligned to the x-axis.When the measurement is triggered, the y-axis of the worktable is automatically operated to achieve the scanning motion.The y-axis positioning error, measured through a rotary optical encoder, results to be lower than <5E-3 μm Part program pre-processing.The part program g-code is automatically analysed to retrieve the extrusion path for each layer.Additional code is inserted for the realisation of two sacrificial structures at the sides of the layer: these are needed to reinstate steady-state extrusion conditions after the extrusion process is interrupted to measure the completed layer.A geometric analysis of the extrusion path for each layer is also carried out to determine the scan path for the laser line scanner in order to achieve full coverage of the layer.The scan path is automatically converted into new blocks of g-code and inserted in the part program, for execution between consecutive layers.Part program execution.When the FFF machine executes the part program to fabricate the part, the following operations are repeated for each layer.The extrusion path is retrieved from g-code and used by a simulator to estimate the layout of deposited material in the x,y plane of the current layer.Width and length of each deposited strand are computed using previously developed extrusion models [6], taking into account extrusion trajectory, traversal velocity of the extrusion nozzle, and filament feed-rate.The layout of deposited material on the layer is encoded as a raster, binary grid (material vs. voids) of user-defined resolution.The resulting grid is used as the nominal reference of where material is supposed to be after layer fabrication, for use by the in-process inspection system.Once layer fabrication is complete, the FFF machine executes the scanning path encoded in the same part program.The primary purpose of laser line scanning is the identification of transitions between material and void in the current layer.Therefore, instead of delivering a complete 2.5D reconstruction of the layer by triangulation, a simpler and quicker approach is implemented as summarised in Fig. 2. For each position of the laser line, the optical microscope acquires an image from a tilted viewpoint.Parts of the laser line that hit layer material will appear located within a predefined horizontal reference band within the image, whilst parts of the laser line hitting void regions (possibly falling through on material at lower layers) will appear outside and below the reference band.A simple binarization process within the image band can be adopted to generate a binary strip encoding material and void regions within the band covered by the laser line.As the imaging system is calibrated, image processing results (in pixels) can be converted to mm.

Figure 2. Operating principle of the laser line scanner; a) digital image acquired by the microscope. Laser line regions hitting layer material are located within the reference band. Laser line regions hitting void or lower layers are located outside and below; b) resulting binary strip for the laser line shown (shown in pixel units).
The binarization approach implemented in the laser line scanning method proves to be more robust at finding material boundaries compared to edge detection in 2D images, as it is not as easily confused by shadows or focusing issues, although the analysis of the laser line carries its own series of issues and challenges.Regardless, an entire binary raster map of the layer material/void regions can be obtained by combining strips generated during scanning.In-process inspection then consists of comparing the binary, raster map obtained via simulation (layout of deposited material estimated from g-code) with the binary, raster map obtained via laser line scanning (measure of actual deposited material).Note that, whilst the resolution of the measured raster map is constrained by the properties of the scanning device, the resolution of the simulated raster map is only limited by computational resources, therefore the latter is currently downsampled to 0.1 mm × 0.1 mm to match the resolving power of the measurement solution, so that a one-to-one "pixelwise" comparison is possible.Since the laser scanning device is rigidly coupled with the extrusion system and positioned using the same axes and encoders, simulated and measured layer data are intrinsically registered through an offset calculated during calibration of the laser scanning device.Pixel-wise discrepancies calculated in the inspection procedure are then highlighted as excess or missing material pixels (in the measurement map, with respect to the simulated reference), and can be used as the basis to develop in-process monitoring solutions.Finally, if needed, both the simulated and measured raster maps can be vertically stacked into voxel volumes (using layer thickness as voxel z-width), to respectively generate the volumetric nominal expectation for the part and the actual volumetric measurement of the fabricated part.These models can be compared voxel-wise to infer further quality issues, although post-process.

Results
The test case chosen for this work is illustrated in Fig. 3.It consists of a test geometry featuring an internal lattice-like reticular structure.The process to generate the binary, raster map of material layout by simulation is illustrated in Fig. 4 for one of the layers.Within the in-process inspection solution, this map is effectively a digital twin of the current layer.The result of laser line scanning for the same layer and the comparison map (measured vs simulated) are illustrated in Fig. 5.  4. Red pixels: excess material (material present in measurement, but not in the simulated map), blue pixels: missing material (material absent in measurement, but present in the simulated map), green pixels: correct material (present in both measurement and simulation), transparent pixels: correct voids (absent in both measurement and simulation).
From the analysis of excess and missing material pixels, warning signals can be triggered right after layer measurement, for example based on discrepancy count or spatial distribution.The actual triggering criteria are application-dependent.In Fig. 6 the two three-dimensional voxel models obtained by vertical stacking of simulation and measurement raster maps are shown.Three-dimensional maps of missing and excess material voxels can be generated as well, in case post-process assessment of part quality is needed.The three-dimensional distribution of detected discrepancies may provide further avenues for quality assessment.

Conclusions
An original solution for in-process inspection of fused filament fabrication has been proposed, capable of addressing the realisation of complex geometries featuring hollow and reticular internal features, such as lattice patterns.The in-process inspection solution combines optical tomography, implemented via laser line scanning, with a pixel and voxel-based method for real-time comparison between the fabricated geometry and a nominal reference generated by simulation (digital twin).The main points of strength of the proposed solution are in the original use of laser line scanning to retrieve external and internal layer contours in optical tomography, which overcomes the limitations of edge detection applied to 2D digital images; and the pixel-/ voxelbased modelling and comparison of expected and measured geometries, which allows for robust detection of missing and excess material.Further challenges to be addressed are related to determining the performance of the inspection method.The main issue is that, at the moment, we haven't fully characterised the metrological performance of the laser line scanning-based measurement solution, therefore we are unable to quantify how much of the observed discrepancies may be due to measurement error.Metrological characterisation is challenging due to the absence of a reference measurement result to compare our solution against.The use of Xray CT-scanning is being considered, but CT-scanning itself is hardly a reference, given the currently unsolved issues related to boundary determination and effects of measurement artefacts (e.g., beam-hardening).Moreover, CT-scanning would have to be performed post-process, on a final part which could have changed shape because of warpage, shrinkage or hydrophilic effects.
The second main unsolved issue in reference to the pixel-wise comparison lays on the simulation side, as part of the observed discrepancies may be indeed due to the simulation's inability to accurately reproduce the expected results, for example in relation to the expected thickness of the deposited strand.Work is in progress to further refine and validate the simulation model.Other issues which may influence the comparison are related to the data processing steps and include for example the process of binarising the laser line position in measurement (affected by multiple factors, including the determination of the laser centreline and the determination of the vertical width of the reference band), as well as the steps of rasterization/voxelization, in particular the resolutions adopted for comparing both layer maps and entire 3D volumes.The assessment of inspection performance in relation to the sizes of the most frequent geometric anomalies in FFFmanufactured goods (application dependent) will also need to be addressed.
Figure 1.a) FFF machine prototype; b) laser line scanner.

Figure 3 .
Test case.Material: PLA, approx.size: 20 mm x 8 mm x 20 mm.Lattice structure consisting of a network of struts arranged within cubic cells (5 mm size) and strut thickness equal to 1.2 mm; a) CAD model; b) one of the fabricated specimens, along with the sacrificial structures at the sides.

Figure 4 .
Process to generate (by simulation) the expected binary, raster map of material layout for one layer of the test part; a) extrusion path as extracted from the g-code; b) material layout in the x,y plane using a simulation model to estimate length and width of the deposited strands; c) final material/void raster map, resampled at the resolution of the measured map.a) b) Figure 5. a) raster map resulting from laser line scanning measurement; b) pixel-wise comparison of the measurement map with the simulated map previously shown in Fig.

Figure 6 .
Voxel models of simulated and measured part and voxel-wise analysis of discrepancies.a) voxel model of the simulated part; b) voxel model of the measured part; c) three-dimensional distribution of voxels representing excess material; d) three-dimensional distribution of voxels representing missing material.