Soft-tissue surgeries, such as tumor resections, are complicated by tissue deformations that can obscure the accurate location and shape of tissues. By representing tissue surfaces as point clouds and applying non-rigid point cloud registration (PCR) methods, surgeons can better understand tissue deformations before, during, and after surgery. Existing non-rigid PCR methods, such as feature-based approaches, struggle with robustness against challenges like noise, outliers, partial data, and large deformations, making accurate point correspondence difficult. Although learning-based PCR methods, particularly transformer-based approaches, have recently shown promise due to their attention mechanisms for capturing interactions, their robustness remains limited in challenging scenarios. In this paper, we present DefTransNet, a novel end-to-end transformer-based architecture for non-rigid PCR. DefTransNet is designed to address the key challenges of deformable registration, including large deformations, outliers, noise, and partial data, by inputting source and target point clouds and outputting displacement vector fields. The proposed method incorporates a learnable transformation matrix to enhance robustness to affine transformations, integrates global and local geometric information, and captures long-range dependencies among points using transformers. We validate our approach on four datasets: ModelNet, SynBench, 4DMatch, and DeformedTissue, using both synthetic and real-world data to demonstrate the generalization of our proposed method. Experimental results demonstrate that DefTransNet outperforms current state-of-the-art registration networks across various challenging conditions.

Tissue deformation is a critical issue in soft-tissue surgery, particularly during tumor resection, as it causes landmark displacement, complicating tissue orientation. The authors conducted an experimental study on 45 pig head cadavers to simulate tissue deformation, approved by the Mannheim Veterinary Office (DE 08 222 1019 21). We used 3D cameras and head-mounted displays to capture tissue shapes before and after controlled deformation induced by heating. The data were processed using software such as Meshroom, MeshLab, and Blender to create and evaluate 2½D meshes. The dataset includes different levels of deformation, noise, and outliers, generated using the same approach as the SynBench dataset. 1. Deformation_Level: 10 different deformation levels are considered. 0.1 and 0.7 are representing minimum and maximum deformation, respectively. Source and target files are available in each folder. The deformation process is just applied to target files. For simplicity, the corresponding source files to the target ones are available in this folder with the same name, but source ones start with Source_ and the target files start with Target_. The number after Source_ and Target_ represents the primitive object in the “Data” folder. For example, Target_3 represents that this file is generated from object number 3 in the “Data” folder. The two other numbers in the file name represent the percentage number of control points and the width of the Gaussian radial basis function, respectively. 2. Noisy_Data For all available files in the “Deformation_Level” folder (for all deformation levels), Noisy data is generated. They are generated in 4 different noise levels namely, 0.01, 0.02, 0.03, and 0.04 (More explanation about implementation can be found in the paper). The name of the files is the same as the files in the “Deformation_Level” folder. 3. Outlier_Data For all available files in the “Deformation_Level” folder (for all deformation levels), data with outliers is generated. They are generated in different outlier levels, in 5 categories, namely, 5%, 15%, 25%, 35%, and 45% (More explanation about implementation can be found in the paper). The name of the files is the same as the files in the “Deformation_Level” folder. Furthermore, for each file, there is one additional file with the same name but is started with “Outlier_”. This represents a matrix with the coordinates of outliers. Then, it would be possible to use these files as benchmarks to check the validity of future algorithms. Additional notes: Considering the fact that all challenges are generated under small to large deformation levels, the DeformedTissue dataset makes it possible for users to select their desired data based on the ability of their proposed method, to show how robust to complex challenges their methods are. The DeformedTissue dataset is publicly available under https://doi.org/10.11588/DATA/OAUXWS.

Despite the existence of several datasets for deformable point cloud registration, the absence of a comprehensive benchmark with all challenges makes it difficult to achieve fair evaluations among different methods. SynBench, a new non-rigid point cloud registration dataset created using SimTool—a toolset for soft body simulation in Flex and Unreal Engine. SynBench provides the ground truth of corresponding points between two point sets and encompasses key registration challenges, including varying levels of deformation, noise, outliers, and incompleteness. To the best of the authors’ knowledge, compared to existing datasets, SynBench possesses three particular characteristics: 1) it is the first benchmark that provides various challenges for non-rigid point cloud registration, 2) SynBench encompasses challenges of varying difficulty levels, 3) It includes ground truth corresponding points both before and after deformation. The authors believe that SynBench makes it possible for future non-rigid point cloud registration methods to present a fair comparison of their achievements. The SynBench is publicly available under https://doi.org/10.11588/data/R9IKCF.

SynBench dataset is created using SimTool — a toolset for soft body simulation in Flex and Unreal Engine.

If you use DefTransNet code, please cite:
@article{10.1088/1361-6501/ade613,
author={Monji Azad, Sara and Kinz, Marvin and kothari, Siddharth and Khanna, Robin and Mihan, Amrei Carla and Maennle, David and Scherl, Claudia and Hesser, Juergen W},
title={DefTransNet: A Transformer-based Method for Non-Rigid Point Cloud Registration in the Simulation of Soft Tissue Deformation},
journal={Measurement Science and Technology},
url={http://iopscience.iop.org/article/10.1088/1361-6501/ade613},
year={2025},
publisher={IOP Publishing}
}

If you use DeformedTissue Dataset please cite both the paper and data:
@article{10.1088/1361-6501/ade613,
author={Monji Azad, Sara and Kinz, Marvin and kothari, Siddharth and Khanna, Robin and Mihan, Amrei Carla and Maennle, David and Scherl, Claudia and Hesser, Juergen W},
title={DefTransNet: A Transformer-based Method for Non-Rigid Point Cloud Registration in the Simulation of Soft Tissue Deformation},
journal={Measurement Science and Technology},
url={http://iopscience.iop.org/article/10.1088/1361-6501/ade613},
year={2025},
publisher={IOP Publishing}
}

@data{DATA/OAUXWS_2025,
author = {Monji Azad, Sara and Scherl, Claudia and Männle, David},
publisher = {heiDATA},
title = {{DeformedTissue Dataset}},
year = {2025},
version = {V1},
doi = {10.11588/DATA/OAUXWS},
url = {https://doi.org/10.11588/DATA/OAUXWS}
}

If you use SynBench Dataset please cite both the paper and data:

@article{monji2024robust,
title={Robust-DefReg: a robust coarse to fine non-rigid point cloud registration method based on graph convolutional neural networks},
author={Monji-Azad, Sara and Kinz, Marvin and M{"a}nnel, David and Scherl, Claudia and Hesser, J{"u}rgen},
journal={Measurement Science and Technology},
volume={36},
number={1},
pages={015426},
year={2024},
publisher={IOP Publishing}
}

@data{data/R9IKCF_2023,
author = {Marvin Kinz},
publisher = {heiDATA},
title = {{SimTool-SynBench}},
year = {2023},
version = {V2},
doi = {10.11588/data/R9IKCF},
url = {https://doi.org/10.11588/data/R9IKCF}
}

If you use SimTool, please cite:

@article{monji2023simtool,
title={SimTool: A toolset for soft body simulation using Flex and Unreal Engine},
author={Monji-Azad, Sara and Kinz, Marvin and Hesser, J{"u}rgen and L{"o}w, Nikolas},
journal={Software Impacts},
volume={17},
pages={100521},
year={2023},
publisher={Elsevier}
}

To run this code locally or on a server, it is highly recommended to isolate the dependencies using a Python virtual environment.

If you are running this on a headless Linux server or within Windows Subsystem for Linux (WSL), the visualization libraries (Plotly/Kaleido) require specific shared graphical libraries to render and save 3D plots to PDF.

Run the following to install the necessary system dependencies:

sudo apt update && sudo apt upgrade -y
sudo apt install python3-venv python3-pip -y
sudo apt install libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 libxrandr2 libgbm1 libasound2t64 -y

Create and activate an isolated environment:

python3 -m venv deftrans_env
source deftrans_env/bin/activate

Note for Beginners: A virtual environment only remains active as long as your current terminal window is open. If you close your terminal, restart your computer, or open a new tab, you must reactivate the environment by navigating to the project folder and running source deftrans_env/bin/activate again before executing any code. You will know it is active when (deftrans_env) appears at the beginning of your terminal prompt.

To leverage GPU acceleration, install the CUDA-enabled version of PyTorch. (The command below is for CUDA 12.1; adjust the URL if your hardware requires a different version):

pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Install the remaining mathematical, visualization, and dataset utilities:

pip install -r requirements.txt

This project utilizes the ModelNet10 dataset provided by Princeton University. You can download and extract it directly into the project root using standard command-line tools:

# Download the dataset zip file
wget http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip

# Extract the contents (requires the 'unzip' package)
unzip ModelNet10.zip

# Clean up the downloaded zip archive
rm ModelNet10.zip

Note: Ensure the extracted folder is named ModelNet10 and sits in the same directory as the Jupyter notebook.

With your environment active and dataset downloaded, you can start the Jupyter Lab server to open and run DefTransNet.ipynb.

Start the server using the following command:

jupyter lab --no-browser

The terminal will output a URL (usually starting with http://localhost:8888/). Copy and paste this link into your web browser to open the workspace.

This notebook uses Plotly and Kaleido to generate interactive 3D graphs and save them as PDF files during the evaluation phase. You must configure the notebook based on the operating system you are using:

• For Headless Environments (WSL, Remote Servers, SSH): Headless environments lack the native display drivers required to silently open Chrome and render PDFs, which will cause the training loop to crash. To fix this, go to Cell 6 in the notebook and change the visualization flag from 1 to 0:

# Change this:
net = DefTransNet(show=1)

# To this:
net = DefTransNet(show=0)

This safely disables the PDF generation, allowing the math and training loops to run at full speed.

• For Desktop Environments (Ubuntu Desktop, Windows natively, etc.): If you are running the code on an operating system with a standard graphical user interface and Google Chrome installed, you can leave the code exactly as is (show=1). The notebook will successfully generate and save the 3D visual evaluations into a Plots/ directory.