Bug: Problem mounting symlinks with runc 1.3.3 #8464
Description
Description:
After upgrade to runc-1.3.3 SAM is not able to build the code using --use-container --mount-symlinks options.
Build container exits with an error:
failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting \"/project-dir/function_dir\" to rootfs at \"/tmp/samcli/source/function_dir\": create mountpoint for /tmp/samcli/source/function_dir mount: make mountpoint \"/tmp/samcli/source/function_dir\": openat2 /var/lib/docker/overlay2/xxx/merged/tmp/samcli/source/function_dir: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"
I found out, that on earlier runc version, it works as expected. I was able to workaround the issue, by downgrading runc to version 1.1.0-0ubuntu1.
Observed result:
...
Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
2025-11-28 14:16:36,816 | Mounting /project-dir/function-dir as
/tmp/samcli/source:ro,delegated, inside runtime container
2025-11-28 14:16:36,820 | Mounting resolved symlink
(/project-dir/function-dir/symbolic_link_path -> /project-dir/function-dir/symbolic_link_path) as
/tmp/samcli/source/symbolic_link_path:ro,delegated, inside runtime container
2025-11-28 14:16:36,823 | ContainerClientFactory.create_client() called
2025-11-28 14:16:36,825 | Admin preference: None
2025-11-28 14:16:36,826 | Using auto-detected client creation
2025-11-28 14:16:36,828 | Trying Docker client creation
2025-11-28 14:16:36,830 | Creating Docker container client from environment variable.
2025-11-28 14:16:36,833 | Creating container client with parameters: {'version': '1.35'}
2025-11-28 14:16:36,836 | DockerContainerClient created successfully
2025-11-28 14:16:36,842 | Using Docker as Container Engine.
2025-11-28 14:16:36,843 | Set global container socket path: container_socket_path=
2025-11-28 14:16:36,904 | SAM_CONTAINER_ID:
xxxxxx
...hangs here
Expected result:
Successful lambda code build.
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
SAM version: 1.148.0
My /etc/os-release:
NAME="Linux Mint"
VERSION="21.3 (Virginia)"
ID=linuxmint
ID_LIKE="ubuntu debian"
PRETTY_NAME="Linux Mint 21.3"
VERSION_ID="21.3"
HOME_URL="https://www.linuxmint.com/"
SUPPORT_URL="https://forums.linuxmint.com/"
BUG_REPORT_URL="http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/"
PRIVACY_POLICY_URL="https://www.linuxmint.com/"
VERSION_CODENAME=virginia
UBUNTU_CODENAME=jammy
Docker, containerd, runc version set that does work:
||/ Name Version Architecture Description
+++-==============-=======================-============-=================================
ii containerd 1.7.28-0ubuntu1~22.04.1 amd64 daemon to control runC
ii docker.io 28.2.2-0ubuntu1~22.04.1 amd64 Linux container runtime
ii runc 1.1.0-0ubuntu1 amd64 Open Container Project - runtime
The version set which does not work:
||/ Name Version Architecture Description
+++-==============-=======================-============-=================================
ii containerd 1.7.28-0ubuntu1~22.04.1 amd64 daemon to control runC
ii docker.io 28.2.2-0ubuntu1~22.04.1 amd64 Linux container runtime
ii runc 1.3.3-0ubuntu1~22.04.3 amd64 Open Container Project - runtime