Publications (Full Publication List): 2001-2005; 2006-2010; 2011-2015; 2016-2020; 2021; 2022; 2023; Under Review. Disclaimer: The documents listed on this page are copyright-protected. By clicking on the [PDF] links below, you confirm that you or your institution have the right to access the corresponding pdf file. Recent Papers Xingjian Han, Yu Jiang, Weiming Wang, Guoxin Fang, Simeon Gill, Zhiqiang Zhang, Shengfa Wang, Jun Saito, Deepak Kumar, Zhongxuan Luo, Emily Whiting, and Charlie C.L. Wang, "Motion-driven neural optimizer for prophylactic braces made by distributed microstructures", ACM SIGGRAPH Asia 2024, December 3-6, 2024, Tokyo, Japan. [Project] [arXiv] [Video@YouTube] Abstract Joint injuries, and their long-term consequences, present a substantial global health burden. Wearable prophylactic braces are an attractive potential solution to reduce the incidence of joint injuries by limiting joint movements that are related to injury risk. Given human motion and ground reaction forces, we present a computational framework that enables the design of personalized braces by optimizing the distribution of microstructures and elasticity. As varied brace designs yield different reaction forces that influence kinematics and kinetics analysis outcomes, the optimization process is formulated as a differentiable end-to-end pipeline in which the design domain of microstructure distribution is parameterized onto a neural network. The optimized distribution of microstructures is obtained via a self-learning process to determine the network coefficients according to a carefully designed set of losses and the integrated biomechanical and physical analyses. Since knees and ankles are the most commonly injured joints, we demonstrate the effectiveness of our pipeline by designing, fabricating, and testing prophylactic braces for the knee and ankle to prevent potentially harmful joint movements. Yuming Huang, Yuhu Guo, Renbo Su, Xingjian Han, Junhao Ding, Tianyu Zhang, Tao Liu, Weiming Wang, Guoxin Fang, Xu Song, Emily Whiting, and Charlie C.L. Wang, "Learning based toolpath planner on diverse graphs for 3D printing", ACM Transactions on Graphics (SIGGRAPH Asia 2024), vol.43, no.6, article no.229 (16 pages), December 2024. [Project] [arXiv] [Video@YouTube] Abstract This paper presents a learning based planner for computing optimized 3D printing toolpaths on prescribed graphs, the challenges of which include the varying graph structures on different models and the large scale of nodes & edges on a graph. We adopt an on-the-fly strategy to tackle these challenges, formulating the planner as a Deep Q-Network (DQN) based optimizer to decide the next 'best' node to visit. We construct the state spaces by the Local Search Graph (LSG) centered at different nodes on a graph, which is encoded by a carefully designed algorithm so that LSGs in similar configurations can be identified to re-use the earlier learned DQN priors for accelerating the computation of toolpath planning. Our method can cover different 3D printing applications by defining their corresponding reward functions. Toolpath planning problems in wire-frame printing, continuous fiber printing, and metallic printing are selected to demonstrate its generality. The performance of our planner has been verified by testing the resultant toolpaths in physical experiments. By using our planner, wire-frame models with up to 4.2k struts can be successfully printed, up to 93.3% of sharp turns on continuous fiber toolpaths can be avoided, and the thermal distortion in metallic printing can be reduced by 24.9%. Xiangjia Chen, Lip M. Lai, Zishun Liu, Chengkai Dai, Isaac C.W. Leung, Charlie C.L. Wang, and Yeung Yam, "Computer-controlled 3D surface weaving", Robotics and Computer-Integrated Manufacturing, vol.90, 102819 (12 pages), December 2024. [arXiv] [Video@YouTube] Abstract In this paper, we present a new computer-controlled weaving technology that enables the fabrication of woven structures in the shape of given 3D surfaces by using threads in non-traditional materials with high bending-stiffness, allowing for multiple applications with the resultant woven fabrics. A new weaving machine and a new manufacturing process are developed to realize the function of 3D surface weaving by the principle of short-row shaping. A computational solution is investigated to convert input 3D freeform surfaces into the corresponding weaving operations (indicated as W-code) to guide the operation of this system. A variety of examples using cotton threads, conductive threads and optical fibres are fabricated by our prototype system to demonstrate its functionality. Tao Liu, Tianyu Zhang, Yongxue Chen, Yuming Huang, and Charlie C.L. Wang, "Neural slicer for multi-axis 3D printing", ACM Transactions on Graphics (SIGGRAPH 2024), vol.43, no.4, article no.85 (15 pages), July 2024. [Project] [arXiv] [Source Code] [Video@YouTube] Abstract We introduce a novel neural network-based computational pipeline as a representation-agnostic slicer for multi-axis 3D printing. This advanced slicer can work on models with diverse representations and intricate topology. The approach involves employing neural networks to establish a deformation mapping, defining a scalar field in the space surrounding an input model. Isosurfaces are subsequently extracted from this field to generate curved layers for 3D printing. Creating a differentiable pipeline enables us to optimize the mapping through loss functions directly defined on the field gradients as the local printing directions. New loss functions have been introduced to meet the manufacturing objectives of support-free and strength reinforcement. Our new computation pipeline relies less on the initial values of the field and can generate slicing results with significantly improved performance. Yingjun Tian, Guoxin Fang, Renbo Su, Weiming Wang, Simeon Gill, Andrew Weightman, and Charlie C.L. Wang, "Function based sim-to-real learning for shape control of deformable free-form surfaces", Robotics: Science and Systems Conference, July 15-19, 2024, Delft, Netherlands. [arXiv] [Video@YouTube] Abstract For the shape control of deformable free-form surfaces, simulation plays a crucial role in establishing the mapping between the actuation parameters and the deformed shapes. The differentiation of this forward kinematic mapping is usually employed to solve the inverse kinematic problem for determining the actuation parameters that can realize a target shape. However, the free-form surfaces obtained from simulators are always different from the physically deformed shapes due to the errors introduced by hardware and the simplification adopted in physical simulation. To fill the gap, we propose a novel deformation function based sim-to-real learning method that can map the geometric shape of a simulated model into its corresponding shape of the physical model. Unlike the existing sim-to-real learning methods that rely on completely acquired dense markers, our method accommodates sparsely distributed markers and can resiliently use all captured frames - even for those in the presence of missing markers. To demonstrate its effectiveness, our sim-to-real method has been integrated into a neural network-based computational pipeline designed to tackle the inverse kinematic problem on a pneumatically actuated deformable mannequin. Yinan Meng, Guoxin Fang, Jiong Yang, Yuhu Guo, and Charlie C.L. Wang, "Spring-IMU fusion based proprioception for feedback control of soft manipulators", IEEE/ASME Transactions on Mechatronics, Focused Section on Compliant Mechanisms for Mechatronics, vol.29, no.2, pp.832-842, April 2024. [arXiv] [Video@YouTube] Abstract This paper presents a novel framework to realize proprioception and closed-loop control for soft manipulators. Deformations with large elongation and large bending can be precisely predicted using geometry-based sensor signals obtained from the inductive springs and the inertial measurement units (IMUs) with the help of machine learning techniques. Multiple geometric signals are fused into robust pose estimations, and a data-efficient training process is achieved after applying the strategy of sim-to-real transfer. As a result, we can achieve proprioception that is robust to the variation of external loading and has an average error of 0.7% across the workspace on a pneumatic-driven soft manipulator. The realized proprioception on soft manipulator is then contributed to building a sensor-space based algorithm for closed-loop control. A gradient descent solver is developed to drive the end-effector to achieve the required poses by iteratively computing a sequence of reference sensor signals. A conventional controller is employed in the inner loop of our algorithm to update actuators (i.e., the pressures in chambers) for approaching a reference signal in the sensor-space. The systematic function of closed-loop control has been demonstrated in tasks like path following and pick-and-place under different external loads. Guoxin Fang, Tianyu Zhang, Yuming Huang, Zhizhou Zhang, Kunal Masania, and Charlie C.L. Wang, "Exceptional mechanical performance by spatial printing with continuous fiber: curved slicing, toolpath generation and physical verification", Additive Manufacturing, vol.82, 104048 (16 pages), February 2024. [arXiv] [Video@YouTube] [Project Page] Abstract This work explores a spatial printing method to fabricate continuous fiber-reinforced thermoplastic composites (CFRTPCs), which can achieve exceptional mechanical performance. For models giving complex 3D stress distribution under loads, typical planar-layer based fiber placement usually fails to provide sufficient reinforcement due to their orientations being constrained to planes. The effectiveness of fiber reinforcement could be maximized by using multi-axis additive manufacturing (MAAM) to better control the orientation of continuous fibers in 3D-printed composites. Here, we propose a computational approach to generate 3D toolpaths that satisfy two major reinforcement objectives: 1) following the maximal stress directions in critical regions and 2) connecting multiple load-bearing regions by continuous fibers. Principal stress lines are first extracted in an input solid model to identify critical regions. Curved layers aligned with maximal stresses in these critical regions are generated by computing an optimized scalar field and extracting its iso-surfaces. Then, topological analysis and operations are applied to each curved layer to generate a computational domain that preserves fiber continuity between load-bearing regions. Lastly, continuous fiber toolpaths aligned with maximal stresses are generated on each surface layer by computing an optimized scalar field and extracting its iso-curves. A hardware system with dual robotic arms is employed to conduct the physical MAAM tasks depositing polymer or fiber reinforced polymer composite materials by applying a force normal to the extrusion plane to aid consolidation. When comparing to planar-layer based printing results in tension, up to 644% failure load and 240% stiffness are observed on shapes fabricated by our spatial printing method. Mian Qin, Junhao Ding, Shuo Qu, Xu Song, Charlie C.L. Wang, and Wei-Hsin Liao, "Deep reinforcement learning based toolpath generation for thermal uniformity in laser powder bed fusion process", Additive Manufacturing, vol.79, 103937 (12 pages), January 2024. [arXiv] Abstract Laser powder bed fusion (LPBF) is a widely used metal additive manufacturing technology. However, the accumulation of internal residual stress during printing can cause significant distortion and potential failure. Although various scan patterns have been studied to reduce possible accumulated stress, such as zigzag scanning vectors with changing directions or a chessboard-based scan pattern with divided small islands, most conventional scan patterns cannot significantly reduce residual stress. The proposed adaptive toolpath generation (ATG) algorithms, aiming to minimize the thermal gradients, may result in extremely accumulated temperature fields in some cases. To address these issues, we developed a deep reinforcement learning (DRL)-based toolpath generation framework, with the goal of achieving uniformly distributed heat and avoiding extremely thermal accumulation regions during the LPBF process. Neelotpal Dutta, Tianyu Zhang, Guoxin Fang, Ismail E. Yigit, and Charlie C.L. Wang, "Vector field based volume peeling for multi-axis machining", ASME Journal of Computing and Information Science in Engineering, vol.24, no.5, 051001 (12 pages), May 2024. [arXiv] [Video@YouTube] [Source Code] (This is an extended version of the paper IDETC2023-116742, which gains the Best Paper Award and is published in the ASME IDETC/CIE 2023 Conference, Boston, USA, August 20-23, 2023.) Abstract This paper presents an easy-to-control volume peeling method for multi-axis machining based on the computation taken on vector fields. The current scalar field based methods are not flexible and the vector-field based methods do not guarantee the satisfaction of the constraints in the final results. We first conduct an optimization formulation to compute an initial vector field that is well aligned with those anchor vectors specified by users according to different manufacturing requirements. The vector field is further optimized to be an irrotational field so that it can be completely realized by a scalar field's gradients. Iso-surfaces of the scalar field will be employed as the layers of working surfaces for multi-axis volume peeling in the rough machining. Algorithms are also developed to remove and process singularities of the fields. Our method has been tested on a variety of models and verified by physical experimental machining. Yingjun Tian, Renbo Su, Xilong Wang, Nur Banu Altin, Guoxin Fang, and Charlie C.L. Wang, "OpenPneu: Compact platform for pneumatic actuation with multi-channels", IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2023), Seattle, USA, June 27 - July 1, 2023. [arXiv] [Video@YouTube] [Project Page] Abstract This paper presents a compact system, OpenPneu, to support the pneumatic actuation for multi-chambers on soft robots. Micro-pumps are employed in the system to generate airflow and therefore no extra input as compressed air is needed. Our system conducts modular design to provide good scalability, which has been demonstrated on a prototype with ten air channels. Each air channel of OpenPneu is equipped with both the inflation and the deflation functions to provide a full range pressure supply from positive to negative with a maximal flow rate at 1.7 L/min. High precision closed-loop control of pressures has been built into our system to achieve stable and efficient dynamic performance in actuation. An open-source control interface and API in Python are provided. We also demonstrate the functionality of OpenPneu on three soft robotic systems with up to 10 chambers. Tianyu Zhang, Yuming Huang, Piotr Kukulski, Neelotpal Dutta, Guoxin Fang, and Charlie C.L. Wang, "Support generation for robot-assisted 3D printing with curved layers", IEEE International Conference on Robotics and Automation (ICRA 2023), London, United Kingdom, May 29 - June 2, 2023. [arXiv] [Video@YouTube] Abstract Robot-assisted 3D printing has drawn a lot of attention by its capability to fabricate curved layers that are optimized according to different objectives. However, the support generation algorithm based on a fixed printing direction for planar layers cannot be directly applied for curved layers as the orientation of material accumulation is dynamically varied. In this paper, we propose a skeleton-based support generation method for robot-assisted 3D printing with curved layers. The support is represented as an implicit solid so that the problems of numerical robustness can be effectively avoided. The effectiveness of our algorithm is verified on a dual-material printing platform that consists of a robotic arm and a newly designed dual-material extruder. Experiments have been successfully conducted on our system to fabricate a variety of freeform models. Zhishun Liu, Eugeni L. Doubrovski, Jo M.P. Geraedts, Wenting Wang, Yeung Yam, and Charlie C.L. Wang, "Photogrammetric reconstruction of a stolen statue", Eurographics 2023, Short Paper, Saarbrücken, Germany, May 8-12, 2023. [PDF] [Video@YouTube] Abstract In this paper, we propose a method to reconstruct a digital 3D model of a stolen/damaged statue using photogrammetric methods. This task is challenging because the number of available photos for a stolen statue is in general very limited - especially the side / back view photos. Besides using standard structure-from-motion and multi-view stereo methods, we match image pairs with low overlap using sliding windows and maximize the normalized cross-correlation (NCC) based patch-consistency so that the image pairs can be well aligned into a complete model to build the 3D mesh surface. Our method is based on the prior of the planar side on the statue's pedestal, which can cover a large range of statues. We hope this work will motivate more research efforts for the reconstruction of those stolen / damaged statues and heritage preservation. Yuming Huang, Guoxin Fang, Tianyu Zhang, and Charlie C.L. Wang, "Turning-angle optimized printing path of continuous carbon fiber for cellular structures", Additive Manufacturing, vol.68, 103501 (16 pages), April 2023. [Video@YouTube] Abstract This paper presents an algorithm to compute optimized toolpaths to print thin-walled cellular structures with continuous carbon fiber. The input cellular structure is usually represented by a graph G, and the requirements to ensure high quality manufacturing include the coverage, continuity and sharp-turning prevention of fiber deposition. To satisfy these requirements, we develop a new algorithm to search a continuous path on G's dual graph, the corresponding path on which G minimizes the total energy of tuning angles and can visit all edges of G at least once and at most twice. With the help of a post-processing algorithm, a path determined on the dual graph can be converted into an overlap-reduced toolpath for printing continuous carbon fiber. Physical tests were conducted to demonstrate the advantage of printing paths generated by our algorithm. Up to 46% enhancement on the breaking force is observed during the tensile tests when using slightly less fibers. Weiming Wang, Dongwei Feng, Li Yang, Shan Li, and Charlie C.L. Wang, "Topology optimization of self-supporting lattice structure", Additive Manufacturing, vol.67, 103507 (17 pages), April 2023. [Source Code] Abstract This paper presents a method to generate self-supporting lattice structures in a Topology Optimization (TO) framework. Our method is composed of two phases, which are the TO-based subdivision and the TO-based simplification. Starting from a lattice structure with self-supporting lattice unit cells in a coarse resolution, a subdivision method is proposed to adaptively generate the lattice structure based on the density-based topology optimization framework. The subdivision operators are well-designed to preserve self-supporting property on struts, and a filtering approach is proposed to avoid overhanging nodes. To remove redundant struts on the lattice structures generated by subdivision, a simplification method is developed to satisfy the required volume while a self-supporting constraint is incorporated to ensure the manufacturability of the resultant structures. Our method has been tested on both 2D and 3D examples. Experimental tests demonstrate that our method can effectively generate self-supporting lattice structures with stronger mechanical strength. Mian Qin, Shuo Qu, Junhao Ding, Xu Song, Shiming Gao, Charlie C.L. Wang, Wei-Hsin Liao, "Adaptive toolpath generation for distortion reduction in laser powder bed fusion process", Additive Manufacturing, vol.64, 103432 (15 pages), February 2023. Abstract Laser powder bed fusion (LPBF) is one of the most commonly used metal additive manufacturing technologies due to its advantages in fabricating complex structures. However, traditional scan patterns often cause large distortion due to residual stresses induced during the fabrication process. In this paper, we propose novel adaptive toolpath generation algorithms based on in-situ minimizing thermal gradients so that can reduce the part distortion. The objective function of temperature fields is determined by numerical simulation both considering time and distance. Two constraints including collision-free and smoothing constraints are also incorporated during the toolpath patterns generation process. In addition, to improve the efficiency of adaptive toolpath generation algorithms for large or complex structures, island-based strategy is developed and combined with the adaptive toolpath generation algorithms. Results show that the distortion of our proposed algorithms could reduce about 46% compared with that of the zigzag patterns and reduce about 36% compared with that of the traditional chessboard-based patterns. Tianyu Zhang, Guoxin Fang, Yuming Huang, Neelotpal Dutta, Sylvain Lefebvre, Zekai Murat Kilic, and Charlie C.L. Wang, "S^3-Slicer: A general slicing framework for multi-axis 3D printing", ACM Transactions on Graphics (SIGGRAPH Asia 2022), vol.41, no.6, article no.277 (15 pages), December 2022. (Technical Papers' Best Paper Award) [Project] [PDF] [Source Code] [Video@YouTube] [Conference Webpage] Abstract Multi-axis motion introduces more degrees of freedom into the process of 3D printing to enable different objectives of fabrication by accumulating materials layers upon curved layers. An existing challenge is how to effectively generate the curved layers satisfying multiple objectives simultaneously. This paper presents a general slicing framework for achieving multiple fabrication objectives including support free, strength reinforcement and surface quality. These objectives are formulated as local printing directions varied in the volume of a solid, which are achieved by computing the rotation-driven deformation for the input model. The height field of a deformed model is mapped into a scalar field on its original shape, the isosurfaces of which give the curved layers of multi-axis 3D printing. The deformation can be effectively optimized with the help of quaternion fields to achieve the fabrication objectives. The effectiveness of our method has been verified on a variety of models. Guoxin Fang, Yingjun Tian, Zhi-Xin Yang, Jo M.P. Geraedts, and Charlie C.L. Wang, "Efficient Jacobian-based inverse kinematics with sim-to-real transfer of soft robots by learning", IEEE/ASME Transactions on Mechatronics, vol.27, no.6, pp.5296-5306, December 2022. [arXiv] [Video@YouTube] Abstract This paper presents an efficient learning-based method to solve the inverse kinematic (IK) problem on soft robots with highly non-linear deformation. The major challenge of efficiently computing IK for such robots is due to the lack of analytical formulation for either forward or inverse kinematics. To address this challenge, we employ neural networks to learn both the mapping function of forward kinematics and also the Jacobian of this function. As a result, Jacobian-based iteration can be applied to solve the IK problem. A sim-to-real training transfer strategy is conducted to make this approach more practical. We first generate a large number of samples in a simulation environment for learning both the kinematic and the Jacobian networks of a soft robot design. Thereafter, a sim-to-real layer of differentiable neurons is employed to map the results of simulation to the physical hardware, where this sim-to-real layer can be learned from a very limited number of training samples generated on the hardware. The effectiveness of our approach has been verified on pneumatic-driven soft robots for path following and interactive positioning. Zeyu Zhang, Chenming Wu, Chengkai Dai, Qingqing Shi, Guoxin Fang, Dongfang Xie, Yong-Jin Liu, Charlie C.L. Wang, and Xiu-Jie Wang, "A multi-axis robot-based bioprinting platform for bioactive artificial blood vessel and cardiac tissue fabrication", Bioactive Materials, vol.18, pp.138-150, December 2022. [Video@YouTube] Abstract Despite the recent advances in artificial tissue and organ engineering, how to generate large size viable and functional complex organs still remains as a grand challenge for regenerative medicine. Three-dimensional bioprinting has demonstrated its advantages as one of the major methods in fabricating simple tissues, yet it still faces difficulties to generate vasculatures and preserve somatic cell functions in complex organ production. Here, we overcome the limitations of conventional bioprinting systems by converting a six degree-of-freedom robotic arm into a bioprinter, therefore enables cell printing on 3D complex-shaped vascular scaffolds from all directions. We also developed an oil bath-based cell printing method to better preserve cell natural functions after printing. Together with a self-designed bioreactor and a repeated print-and-culture strategy, our bioprinting system is capable to generate vascularized, contractible, and long-term survived cardiac tissues. Such bioprinting strategy mimics the in vivo organ development process and presents a promising solution for in vitro fabrication of complex organs. Renbo Su, Yingjun Tian, Mingwei Du, and Charlie C.L. Wang, "Optimizing out-of-plane stiffness for soft grippers", IEEE Robotics and Automation Letters, Presented at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Kyoto, Japan, October 23-27, 2022, vol.7, no.4, pp.10430-10437, October 2022. [arXiv] [Video@YouTube] Abstract In this paper, we presented a data-driven framework to optimize the out-of-plane stiffness for soft grippers to achieve mechanical properties as hard-to-twist and easy-to-bend. The effectiveness of this method is demonstrated in the design of a soft pneumatic bending actuator (SPBA). First, a new objective function is defined to quantitatively evaluate the out-of-plane stiffness as well as the bending performance. Then, sensitivity analysis is conducted on the parametric model of an SPBA design to determine the optimized design parameters with the help of Finite Element Analysis (FEA). To enable the computation of numerical optimization, a data-driven approach is employed to learn a cost function that directly represents the out-of-plane stiffness as a differentiable function of the design variables. A gradient-based method is used to maximize the out-of-plane stiffness of the SPBA while ensuring specific bending performance. The effectiveness of our method has been demonstrated in physical experiments taken on 3D-printed grippers. Guoxin Fang, Yingjun Tian, Andrew Weightman, and Charlie C.L. Wang, "Collision-aware fast simulation for soft robots by optimization-based geometric computing", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Kyoto, Japan, October 23-27, 2022. [Source Code] [Video@YouTube] Abstract Soft robots are able to safely interact with environments because of their mechanical compliance. Self-collision is also employed in the modern design of soft robots to enhance their performance in different tasks. However, developing an efficient and reliable simulator which can handle the collision response well, is still a challenging task in the research of soft robotics. This paper presents a collision-aware simulator based on geometric optimization, in which we develop a highly efficient and realistic collision checking / response model incorporating a hyperelastic material property. Both actuated deformation and collision response for soft robots are formulated as geometry-based objectives. The collision-free body of a soft robot can be obtained by minimizing the geometry-based objective function. Different from the FEA-based physical simulation, the proposed pipeline performs a much lower computational cost. Moreover, adaptive remeshing is applied to achieve the improvement of the convergence when dealing with soft robots having large volume variations. Experimental tests are conducted on different soft robots to verify the performance of our approach. Bin Liu, Xiuping Liu, Zhixin Yang, and Charlie C.L. Wang, "Concise and effective network for 3D human modeling from orthogonal silhouettes", ASME Journal of Computing and Information Science in Engineering, vol.22, no.5, 051004 (11 pages), October 2022. [arXiv] [Video@YouTube] [User Study] Abstract In this paper, we revisit the problem of 3D human modeling from two orthogonal silhouettes of individuals (i.e., front and side views). Different from our prior work, a supervised learning approach based on convolutional neural network (CNN) is investigated to solve the problem by establishing a mapping function that can effectively extract features from two silhouettes and fuse them into coefficients in the shape space of human bodies. A new CNN structure is proposed in our work to extract not only the discriminative features of front and side views and also their mixed features for the mapping function. 3D human models with high accuracy are synthesized from coefficients generated by the mapping function. Existing CNN approaches for 3D human modeling usually learn a large number of parameters (from 8M to 350M) from two binary images. Differently, we investigate a new network architecture and conduct the samples on silhouettes as input. As a consequence, more accurate models can be generated by our network with only 2.5M coefficients. The training of our network is conducted on samples obtained by augmenting a publicly accessible dataset. Learning transfer by using datasets with a smaller number of scanned models is applied to our network to enable the function of generating results with gender-oriented (or geographical) patterns. Yingjun Tian, Guoxin Fang, Justas Petrulis, Andrew Weightman, and Charlie C.L. Wang, "Soft robotic mannequin: design and algorithm for deformation control", IEEE/ASME Transactions on Mechatronics, Focused Section on TMECH/AIM Emerging Topics, vol.27, no.4, pp.1820-1828, August 2022. [arXiv] [Video@YouTube] Abstract This paper presents a novel soft robotic system for a deformable mannequin that can be employed to physically realize the 3D geometry of different human bodies. The soft membrane on a mannequin is deformed by inflating several curved chambers using pneumatic actuation. Controlling the freeform surface of a soft membrane by adjusting the pneumatic actuation in different chambers is challenging as the membrane's shape is commonly determined by interaction between all chambers. Using vision feedback provided by a structured-light based 3D scanner, we developed an efficient algorithm to compute the optimized actuation of all chambers which could drive the soft membrane to deform into the best approximation of different target shapes. Our algorithm converges quickly by including the step of pose estimation in the loop of optimization, and the time-consuming step for evaluating derivatives on the deformable membrane is avoided by using the Broyden update when possible. The effectiveness of our soft robotic mannequin with controlled deformation has been verified in experiments. Yabin Xu, Liangliang Nan, Laishui Zhou, Jun Wang, and Charlie C.L. Wang, "HRBF-Fusion: Accurate 3D reconstruction from RGB-D data using on-the-fly implicits", ACM Transactions on Graphics, vol.41, no.3, article no.35 (19 pages), June 2022. [PDF] [arXiv] [Source Code] [Video@YouTube] Abstract Reconstruction of high-fidelity 3D objects or scenes is a fundamental research problem. Recent advances in RGB-D fusion have demonstrated the potential of producing 3D models from consumer-level RGB-D cameras. However, due to the discrete nature and limited resolution of their surface representations (e.g., point- or voxel-based), existing approaches suffer from the accumulation of errors in camera tracking and distortion in the reconstruction, which leads to an unsatisfactory 3D reconstruction. In this paper, we present a method using on-the-fly implicits of Hermite Radial Basis Functions (HRBFs) as a continuous surface representation for camera tracking in an existing RGB-D fusion framework. Furthermore, curvature estimation and confidence evaluation are coherently derived from the inherent surface properties of the on-the-fly HRBF implicits, which devote to a data fusion with better quality. We argue that our continuous but on-the-fly surface representation can effectively mitigate the impact of noise with its robustness and constrain the reconstruction with inherent surface smoothness when being compared with discrete representations. Experimental results on various real-world and synthetic datasets demonstrate that our HRBF-fusion outperforms the state-of-the-art approaches in terms of tracking robustness and reconstruction accuracy. Tim Kuipers, Renbo Su, Jun Wu, and Charlie C.L. Wang, "ITIL: Interlaced topologically interlocking lattice for continuous dual-material extrusion", Additive Manufacturing, vol.50, 102495 (11 pages), February 2022. Abstract We propose a simple and effective 3D lattice consisting of interlaced horizontal beams in vertically alternating directions which interlock topologically: the interlaced topologically interlocking lattice (ITIL). It ensures continuous extrusion and ensures an interlock even for highly flexible materials. We develop analytical models for optimizing the ultimate tensile strength of the ITIL lattice in two different orientations relative to the interface: straight and diagonal. The analytical models are applied to polypropylene (PP) and polylactic acid (PLA) and verified by finite elements method (FEM) simulations and physical tensile experiments. In the diagonal orientation ITIL can obtain 82% of the theoretical upper bound of 8.6MPa. ITIL seems to perform comparably to dovetail interlocking designs, while it lends itself to application to non-vertical interfaces. Optimizing the lattice for non-vertical interfaces, however, remains future work. Xiangjia Chen, Guoxin Fang, Wei-Hsin Liao, and Charlie C.L. Wang, "Field-based toolpath generation for 3D printing continuous fibre reinforced thermoplastic composites", Additive Manufacturing, vol.49, 102470 (13 pages), January 2022. [Dataset of Examples] [Video@YouTube] Abstract We present a field-based method of toolpath generation for 3D printing continuous fibre reinforced thermoplastic composites. Our method employs the strong anisotropic material property of continuous fibres by generating toolpaths along the directions of tensile stresses in the critical regions. Moreover, the density of toolpath distribution is controlled in an adaptive way proportionally to the values of stresses. Specifically, a vector field is generated from the stress tensors under given loads and processed to have better compatibility between neighboring vectors. An optimal scalar field is computed later by making its gradients approximate the vector field. After that, isocurves of the scalar field are extracted to generate the toolpaths for continuous fibre reinforcement, which are also integrated with the boundary conformal toolpaths in user selected regions. The performance of our method has been verified on a variety of models in different loading conditions. Experimental tests are conducted on specimens by 3D printing continuous carbon fibres (CCF) in a polylactic acid (PLA) matrix. Compared to reinforcement by load-independent toolpaths, the specimens fabricated by our method show up to 71.4% improvement on the mechanical strength in physical tests when using the same (or even slightly smaller) amount of continuous fibres. Shengjun Liu, Tao Liu, Qiang Zou, Weiming Wang, Eugeni L. Doubrovski, and Charlie C.L. Wang, "Memory-efficient modeling and slicing of large-scale adaptive lattice structures", ASME Journal of Computing and Information Science in Engineering, vol.21, no.6, 061003 (16 pages), December 2021. [arXiv] [Source Code] Abstract Lattice structures have been widely used in various applications of additive manufacturing due to its superior physical properties. If modeled by triangular meshes, a lattice structure with huge number of struts would consume massive memory. This hinders the use of lattice structures in large-scale applications (e.g., to design the interior structure of a solid with spatially graded material properties). To solve this issue, we propose a memory-efficient method for the modeling and slicing of adaptive lattice structures. A lattice structure is represented by a weighted graph where the edge weights store the struts' radii. When slicing the structure, its solid model is locally evaluated through convolution surfaces and in a streaming manner. As such, only limited memory is needed to generate the toolpaths of fabrication. Also, the use of convolution surfaces leads to natural blending at intersections of struts, which can avoid the stress concentration at these regions. We also present a computational framework for optimizing supporting structures and adapting lattice structures with prescribed density distributions. The presented methods have been validated by a series of case studies with large number (up to 100M) of struts to demonstrate its applicability to large-scale lattice structures. Tianyu Zhang, Xiangjia Chen, Guoxin Fang, Yingjun Tian, and Charlie C.L. Wang, "Singularity-aware motion planning for multi-axis additive manufacturing", IEEE Robotics and Automation Letters, Presented at IEEE International Conference on Automation Science and Engineering (CASE 2021), Lyon, France, August 23-27, 2021, vol.6, no.4, pp.6172-6179, October 2021. (Finalist of Best Student Paper Award) [arXiv] [Source Code] [Video@YouTube] Abstract Multi-axis additive manufacturing enables high flexibility of material deposition along dynamically varied directions. The Cartesian motion platforms of these machines include three parallel axes and two rotational axes. Singularity on rotational axes is a critical issue to be tackled in motion planning for ensuring high quality of manufacturing results. The highly nonlinear mapping in the singular region can convert a smooth toolpath with uniformly sampled waypoints defined in the model coordinate system into a highly discontinuous motion in the machine coordinate system, which leads to over-extrusion / under-extrusion of materials in filament-based additive manufacturing. The problem is challenging as both the maximal and the minimal speeds at the tip of a printer head must be controlled in motion. Moreover, collision may occur when sampling-based collision avoidance is employed. In this paper, we present a motion planning method to support the manufacturing realization of designed toolpaths for multi-axis additive manufacturing. Problems of singularity and collision are considered in an integrated manner to improve the motion therefore the quality of fabrication. Zishun Liu, Xingjian Han, Yuchen Zhang, Xiangjia Chen, Yukun Lai, Eugeni L. Doubrovski, Emily Whiting, and Charlie C.L. Wang, "Knitting 4D garment with elasticity controlled for body motion", ACM Transactions on Graphics (SIGGRAPH 2021), vol.40, no.4, article no.62 (16 pages), August 2021. [Project] [PDF] [Source Code] [Video@YouTube] Abstract In this paper, we present a new computational pipeline for designing and fabricating 4D garments as knitwear that considers comfort during body movement. This is achieved by careful control of elasticity distribution to reduce uncomfortable pressure and unwanted sliding caused by body motion. We exploit the ability to knit patterns in different elastic levels by single-jersey jacquard (SJJ) with two yarns. We design the distribution of elasticity for a garment by physics-based computation, the optimized elasticity on the garment is then converted into instructions for a digital knitting machine by two algorithms proposed in this paper. Specifically, a graph-based algorithm is proposed to generate knittable stitch meshes that can accurately capture the 3D shape of a garment, and a tiling algorithm is employed to assign SJJ patterns on the stitch mesh to realize the designed distribution of elasticity. The effectiveness of our approach is verified on simulation results and on specimens physically fabricated by knitting machines. Rob B.N. Scharff, Guoxin Fang, Yingjun Tian, Jun Wu, Jo M.P. Geraedts, and Charlie C.L. Wang, "Sensing and reconstruction of 3D deformation on pneumatic soft robots", IEEE/ASME Transactions on Mechatronics, vol.26, no.4, pp.1877-1885, August 2021. [arXiv] [Video@YouTube] Abstract Real-time proprioception is a challenging problem for soft robots, which have almost infinite degrees-of-freedom in body deformation. When multiple actuators are used, it becomes more difficult as deformation can also occur on actuators caused by interaction between each other. To tackle this problem, we present a method in this paper to sense and reconstruct 3D deformation on pneumatic soft robots by first integrating multiple low-cost sensors inside the chambers of pneumatic actuators and then using machine learning to convert the captured signals into shape parameters of soft robots. An exterior motion capture system is employed to generate the datasets for both training and testing. With the help of good shape parameterization, the 3D shape of a soft robot can be accurately reconstructed from signals obtained from multiple sensors. We demonstrate the effectiveness of this approach on two designs of soft robots - a robotic joint and a deformable membrane. After parameterizing the deformation of these soft robots into compact shape parameters, we can effectively train the neural networks to reconstruct the 3D deformation from the sensor signals. The sensing and shape prediction pipeline can run at 50Hz in real-time on a consumer-level device. Junhao Ding, Qiang Zou, Shuo Qu, Paulo Bartolo, Xu Song, and Charlie C.L. Wang, "STL-free design and manufacturing paradigm for high-precision powder bed fusion", CIRP Annals - Manufacturing Technology, vol.70, no.1, pp.167-170, July 2021. [PDF] [Source Code] Abstract High-precision powder bed fusion (PBF), together with highly complex geometries necessitate a much more scalable representation of the geometry and an efficient computational pipeline. This paper presents a new digital design and manufacturing paradigm to solve the scalability and efficiency challenges by using the concept of STL-free workflow. It seamlessly integrates implicit solid modelling for design and direct slicing for manufacturing without any intermediate steps related to STL meshes. The presented paradigm has been validated by two case studies involving complex geometries filled with multiscale triply periodic minimal surfaces (TPMS), which are fabricated by PBF with laser beam size 25μm. Chuhua Xian, Dongjiu Zhang, Chengkai Dai, and Charlie C.L. Wang, "Fast generation of high fidelity RGB-D images by deep-learning with adaptive convolution", IEEE Transactions on Automation Science and Engineering, vol.18, no.3, pp.1328-1340, July 2021. [arXiv] [Source Code & Dataset] Abstract Using the raw data from consumer-level RGB-D cameras as input, we propose a deep-learning based approach to efficiently generate RGB-D images with completed information in high resolution. To process the input images in low resolution with missing regions, new operators for adaptive convolution are introduced in our deep-learning network that consists of three cascaded modules - the completion module, the refinement module and the super-resolution module. The completion module is based on an architecture of encoder-decoder, where the features of input raw RGB-D will be automatically extracted by the encoding layers of a deep neural-network. The decoding layers are applied to reconstruct the completed depth map, which is followed by a refinement module to sharpen the boundary of different regions. For the super-resolution module, we generate RGB-D images in high resolution by multiple layers for feature extraction and a layer for up-sampling. Benefited from the adaptive convolution operators newly proposed in this paper, our results outperform the existing deep-learning based approaches for RGB-D image complete and super-resolution. As an end-to-end approach, high fidelity RGB-D images can be generated efficiently at the rate of around 21 frames per second. Book Chapter and Survey Papers Jian Qin, Fu Hu, Ying Liu, Paul Witherell, Charlie C.L. Wang, David W. Rosen, Timothy Simpson, Yan Lu, Qian Tang, "Research and application of machine learning for additive manufacturing", Additive Manufacturing, vol.52, 102691 (25 pages), April 2022. (Highly Cited Paper - Web of Science) Abstract Additive manufacturing (AM) is poised to bring a revolution due to its unique production paradigm. It offers the prospect of mass customization, flexible production, on-demand and decentralized manufacturing. However, a number of challenges stem from not only the complexity of manufacturing systems but the demand for increasingly complex and high-quality products, in terms of design principles, standardization and quality control. These challenges build up barriers to the widespread adoption of AM in the industry and the in-depth research of AM in academia. To tackle the challenges, machine learning (ML) technologies rise to play a critical role as they are able to provide effective ways to quality control, process optimization, modelling of complex systems, and energy management. Hence, this paper employs a systematic literature review method as it is a defined and methodical way of identifying, assessing, and analysing published literature. Then, a keyword co-occurrence and cluster analysis are employed for analysing relevant literature. Several aspects of AM, including Design for AM (DfAM), material analytics, in situ monitoring and defect detection, property prediction and sustainability, have been clustered and summarized to present state-of-the-art research in the scope of ML for AM. Finally, the challenges and opportunities of ML for AM are uncovered and discussed. Under Review Tianyu Zhang, Tao Liu, Neelotpal Dutta, Yongxue Chen, Renbo Su, Zhizhou Zhang, Weiming Wang, and Charlie C.L. Wang, "Toolpath generation for high density spatial fiber printing guided by principal stresses", under review, October 2024. [arXiv] [Video@YouTube] Abstract While multi-axis 3D printing can align continuous fibers along principal stresses in continuous fiber-reinforced thermoplastic (CFRTP) composites to enhance mechanical strength, existing methods have difficulty generating toolpaths with high fiber coverage. This is mainly due to the orientation consistency constraints imposed by vector-field-based methods and the turbulent stress fields around stress concentration regions. This paper addresses these challenges by introducing a 2-RoSy representation for computing the direction field, which is then converted into a periodic scalar field to generate partial iso-curves for fiber toolpaths with nearly equal hatching distance. To improve fiber coverage in stress-concentrated regions, such as around holes, we extend the quaternion-based method for curved slicing by incorporating winding compatibility considerations. Our proposed method can achieve toolpaths coverage between 87.5% and 90.6% by continuous fibers with 1.1mm width. Models fabricated using our toolpaths show up to 84.6% improvement in failure load and 54.4% increase in stiffness when compared to the results obtained from multi-axis 3D printing with sparser fibers. Weiming Wang, Yanhao Hou, Renbo Su, Weiguang Wang, and Charlie C.L. Wang, "Simultaneous optimized mechanical stiffness and cell growth on scaffold by morphology learning", under review, October 2024. [arXiv] [Video@YouTube] Abstract The morphology of microstructures in scaffolds is a critical factor influencing both mechanical properties and biological performance, such as cell growth, in bone tissue engineering. While metamaterials with optimized mechanical properties, such as hyperelasticity, thermal efficiency, or energy absorption, have been designed through complex microstructural geometry with the help of numerical simulation of multi-physics, optimizing biological performance alongside these physical traits remains a significant challenge. This is primarily because biological performance must be evaluated through laboratory experiments. To address this issue, we propose a novel approach that simultaneously optimizes mechanical stiffness and cell growth effectiveness through a data-driven morphology learning method. Our approach effectively extracts and learns shape patterns from a specifically selected dataset of microstructures recognized for their superior cell growth performance. We then integrate these patterns into a computational framework based on topology optimization, enabling the generation of new microstructures that maintain high levels of cell growth while enhancing mechanical stiffness. Using this morphology learning-based method, we identified a scaffold design for bone tissue engineering that demonstrates a significant improvement in both mechanical stiffness (by 29.69%) and cell proliferation (by 37.05% on Day 7 and 33.30% on Day 14). Renbo Su, Tao Liu, Tianyu Zhang, Yingjun Tian, Romain Hautier, Weiming Wang, and Charlie C.L. Wang, "TPMS-based mechanical bonding structures optimized by FEM with periodic boundary conditions", under review, October 2024. [arXiv] Abstract The mechanical interlocking structures can effectively bond disparate materials that have very weak chemical bonds. Existing approaches adopted structures with limited variation in geometry and topology, which prevented the exploration of mechanical bonding structures with optimal performance. The Representative Volume Element (RVE) modeling approach of Finite Element Method (FEM) is often employed to study the mechanical strength of periodic structures with complicated geometry and topology. However, existing RVE-FEM tools have challenges in handling nonperiodic mesh and partial periodicity, limiting the ability to analyze and optimize such interlocking structures for mechanical bonding. This paper presents a formulation based on Triply Periodic Minimal Surface (TPMS) with hybrid basis functions to explore the design space of interlocking structures, where the structure is optimized by a data-driven method based on Bayesian optimization. This is enabled by developing a new tool to apply Periodic Boundary Conditions (PBC) to RVE-FEM. The effectiveness of our approach in optimizing interlocking structures has been validated through mechanical tests on physical models, combining 3D-printed components with molded materials such as silicone that are challenging to print directly. Yongxue Chen, Tianyu Zhang, Yuming Huang, Tao Liu, and Charlie C.L. Wang, "Co-optimization of tool orientations, kinematic redundancy, and waypoint timing for robot-assisted manufacturing", IEEE Transactions on Automation Science and Engineering, Conditionally Accepted, December 2024. [arXiv] [Source Code] [Video@YouTube] Abstract In this paper, we present a concurrent and scalable trajectory optimization method to improve the quality of robot-assisted manufacturing. Our method simultaneously optimizes tool orientations, kinematic redundancy, and waypoint timing on input toolpaths with large numbers of waypoints to improve kinematic smoothness while incorporating manufacturing constraints. Differently, existing methods always determine them in a decoupled manner. To deal with the large number of waypoints on a toolpath, we propose a decomposition-based numerical scheme to optimize the trajectory in an out-of-core manner, which can also run in parallel to improve the efficiency. Simulations and physical experiments have been conducted to demonstrate the performance of our method in examples of robot-assisted additive manufacturing.