How can one match the walls and floor of a given CapturedRoom ?
The transform.eulerAngles of a floor z & y are always 0 !
And the polygons seems to have a different orientation than the walls.
So how to figure out the rotation and match the one from the walls ?
Delve into the world of graphics and game development. Discuss creating stunning visuals, optimizing game mechanics, and share resources for game developers.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
our app is live, and it appears that since the ios 18 update - the VideoMaterial renders pink / purple color instead of the video (picture attached). the audio is rendered properly.
we found that it occurs on old devices: iPhone 11 & iPhone SE 2020.
I've found this thread of Andy Jazz on stackoverflow:
Steps to Reproduce:
Create a plane for the video screen.
Apply a VideoMaterial using AVPlayerItem.
Anchor the model entity to an ARImageAnchor.
Expected Outcome:
The video should play as a material on the plane in RealityKit.
Actual Outcome:
On iOS 18, the plane appears pink, indicating the VideoMaterial isn’t applied.
What I’ve Tried:
-Verified the video URL is correct.
-Checked that the AVPlayerItem and VideoMaterial are initialised correctly.
-Ensured the AVPlayer is playing the video.
I also tried different formats (mov / mp4 / m4v), and verifying that the video's status is readyToPlay.
any suggestions?
Problem Description
I'm encountering an issue with SCNTechnique where the clearColor setting is being ignored when multiple passes share the same depth buffer. The clear color always appears as the scene background, regardless of what value I set. The minimal project for reproducing the issue: https://www.dropbox.com/scl/fi/30mx06xunh75wgl3t4sbd/SCNTechniqueCustomSymbols.zip?rlkey=yuehjtk7xh2pmdbetv2r8t2lx&st=b9uobpkp&dl=0
Problem Details
In my SCNTechnique configuration, I have two passes that need to share the same depth buffer for proper occlusion handling:
"passes": [
"box1_pass": [
"draw": "DRAW_SCENE",
"includeCategoryMask": 1,
"colorStates": [
"clear": true,
"clearColor": "0 0 0 0" // Expecting transparent black
],
"depthStates": [
"clear": true,
"enableWrite": true
],
"outputs": [
"depth": "box1_depth",
"color": "box1_color"
],
],
"box2_pass": [
"draw": "DRAW_SCENE",
"includeCategoryMask": 2,
"colorStates": [
"clear": true,
"clearColor": "0 0 0 0" // Also expecting transparent black
],
"depthStates": [
"clear": false,
"enableWrite": false
],
"outputs": [
"depth": "box1_depth", // Sharing the same depth buffer
"color": "box2_color",
],
],
"final_quad": [
"draw": "DRAW_QUAD",
"metalVertexShader": "myVertexShader",
"metalFragmentShader": "myFragmentShader",
"inputs": [
"box1_color": "box1_color",
"box2_color": "box2_color",
],
"outputs": [
"color": "COLOR"
]
]
]
And the metal shader used to display box1_color and box2_color with splitting:
fragment half4 myFragmentShader(VertexOut in [[stage_in]],
texture2d<half, access::sample> box1_color [[texture(0)]],
texture2d<half, access::sample> box2_color [[texture(1)]]) {
half4 color1 = box1_color.sample(s, in.texcoord);
half4 color2 = box2_color.sample(s, in.texcoord);
if (in.texcoord.x < 0.5) {
return color1;
}
return color2;
};
Expected Behavior
Both passes should clear their color targets to transparent black (0, 0, 0, 0)
The depth buffer should be shared between passes for proper occlusion
Actual Behavior
Both box1_color and box2_color targets contain the scene background instead of being cleared to transparent (see attached image)
This happens even when I explicitly set clearColor: "0 0 0 0" for both passes
Setting scene.background.contents = UIColor.clear makes the clearColor work as expected, but I need to keep the scene background for other purposes
What I've Tried
Setting different clearColor values - all are ignored when sharing depth buffer
Using DRAW_NODE instead of DRAW_SCENE - didn't solve the issue
Creating a separate pass to capture the background - the background still appears in the other passes
Various combinations of clear flags and render orders
Environment
iOS/macOS, running with "My Mac (Designed for iPad)"
Xcode 16.2
Question
Is this a known limitation of SceneKit when passes share a depth buffer? Is there a workaround to achieve truly transparent clear colors while maintaining a shared depth buffer for occlusion testing?
The core issue seems to be that SceneKit automatically renders the scene background in every DRAW_SCENE pass when a shared depth buffer is detected, overriding any clearColor settings.
Any insights or workarounds would be greatly appreciated. Thank you!
I have code that captures a window and displays a cropped image. The problem is 2 fold. Kit doesn't seem to allow to modify stop and recapture image in window mode to capture a portion of the screen.
So this makes me having to crop and display the cropped image via a published variable. This all works find. But seems to stop after some time.
Using an M1 16gig ram. program is taking less than 100meg of mem with 40-70%cpu as the crow flies.
printing captured success in debug mode and sometimes frame isn't valid so guarding against it.
any ideas on how to improve my strategy?
let dic : [AnyHashable:Any] = [
kCGPDFXRegistryName: "http://www.color.org" as CFString,
kCGPDFXOutputConditionIdentifier: "FOGRA43" as CFString,
kCGPDFContextOutputIntent: "GTS_PDFX" as CFString,
kCGPDFXOutputIntentSubtype: "GTS_PDFX" as CFString,
kCGPDFContextCreateLinearizedPDF: "" as CFString,
kCGPDFContextCreatePDFA: "" as CFString,
kCGPDFContextAuthor: "Placeholder" as CFString,
kCGPDFContextCreator: "Placeholder" as CFString
]
Hello,
Now I would like to export my PDF's as PDF/A. In my opinion, there is also the right option for this under Core Graphics.
Unfortunately, the documentation does not show what is 'kCGPDFContextCreatePDFA' or 'kCGPDFContextLinearizedPDF' for
a stringvalue is required.
What I have already tried: GTS_PDFA1 , PDF/A-1, true as CFString.
(Above my CFDictionary. ...Author e.g are working perfectly.)
In the Finder you can see these two options, which I would also like to implement in my app.
Thank you in advance!
Hi everyone,
I’m developing an iPad app that will be running continuously with the screen always on — similar to a restaurant ordering system.
I understand that some of the newer iPad Pro models are equipped with OLED displays. I'm concerned about the potential risk of screen burn-in due to static UI elements being displayed for extended periods.
Does burn-in occur on the OLED iPad Pro models under such usage?
Would it be advisable to implement a screen saver or periodically animate/change parts of the UI to prevent this?
Any insights or best practices would be greatly appreciated. Thank you!
Topic:
Graphics & Games
SubTopic:
General
I'm using RealityView in my iOS game mxied with SwiftUI. For the following 2 example usages, the simulator will only render the first RealityView, and the second one is either super laggy or show a black model. Running on the real device is all good, just simualtor has this issue.
Have a TabView and each tab has a RealityView.
Have a root view and detail view connected via a push navigation, both root and detail have a RealityView.
In the Simulator, the second RealityView is going to be very choppy and basically unusable, but on a real iPhone everything looks great.
Is this a known simulator issue or I did something bad?
When running my game in the Unity Editor on Windows platform I get an error:
DllNotFoundException: GameKitWrapper assembly:<unknown assembly> type:<unknown type> member:(null)
Apple.GameKit.DefaultNSErrorHandler.Init () (at ./Library/PackageCache/com.apple.unityplugin.gamekit@0abcad546f73/Source/DefaultHandlers.cs:35)
This is because GameKitWrapper dynamically linked library is not available under Windows platform.
Besides, "Apple Build Settings" are declared under UNITY_EDITOR_OSX and also not available under Windows platform.
Does anyone managed to solve this?
We have a macOS app (not yet released, but in use by ourselves), that provides scoreboards for streaming sport events.
Today it is expected, that there are nice animations for goals, etc. We are streaming using NDI, which requires a CVPixelBuffer for each frame.
We currently create these animations using CABasicAnimation, CAAnimation and CAKeyframeAnimation. In addition we use ScreenCaptureKit to generate the frames.
This works fine with 25/30 fps, as long as the window where our animations are performed in is visible. But this is not what it should be. We have a smaller window as main app window and control display performing the animations in reduced size, while the streaming animations need to be in HD format and later maybe in 4K.
When using an offscreen window, the animations are not calculated. We get 1 frame per second or so. So we actually have to connect an external display to the MacBook and open the large windows there. Ugly solution.
Do we use a completely wrong approach? Or is there a way to tell the macOS to perform the animations although it is an offscreen window?
If it cannot work that way, what is an alternative?
Hey, I've been struggling with this for some days now.
I am trying to write to a sparse texture in a compute shader. I'm performing the following steps:
Set up a sparse heap and create a texture from it
Map the whole area of the sparse texture using updateTextureMapping(..)
Overwrite every value with the value "4" in a compute shader
Blit the texture to a shared buffer
Assert that the values in the buffer are "4".
I have a minimal example (which is still pretty long unfortunately).
It works perfectly when removing the line heapDesc.type = .sparse.
What am I missing? I could not find any information that writes to sparse textures are unsupported. Any help would be greatly appreciated.
import Metal
func sparseTexture64x64Demo() throws {
// ── Metal objects
guard let device = MTLCreateSystemDefaultDevice()
else { throw NSError(domain: "SparseNotSupported", code: -1) }
let queue = device.makeCommandQueue()!
let lib = device.makeDefaultLibrary()!
let pipeline = try device.makeComputePipelineState(function: lib.makeFunction(name: "addOne")!)
// ── Texture descriptor
let width = 64, height = 64
let format: MTLPixelFormat = .r32Uint // 4 B per texel
let desc = MTLTextureDescriptor()
desc.textureType = .type2D
desc.pixelFormat = format
desc.width = width
desc.height = height
desc.storageMode = .private
desc.usage = [.shaderWrite, .shaderRead]
// ── Sparse heap
let bytesPerTile = device.sparseTileSizeInBytes
let meta = device.heapTextureSizeAndAlign(descriptor: desc)
let heapBytes = ((bytesPerTile + meta.size + bytesPerTile - 1) / bytesPerTile) * bytesPerTile
let heapDesc = MTLHeapDescriptor()
heapDesc.type = .sparse
heapDesc.storageMode = .private
heapDesc.size = heapBytes
let heap = device.makeHeap(descriptor: heapDesc)!
let tex = heap.makeTexture(descriptor: desc)!
// ── CPU buffers
let bytesPerPixel = MemoryLayout<UInt32>.stride
let rowStride = width * bytesPerPixel
let totalBytes = rowStride * height
let dstBuf = device.makeBuffer(length: totalBytes, options: .storageModeShared)!
let cb = queue.makeCommandBuffer()!
let fence = device.makeFence()!
// 2. Map the sparse tile, then signal the fence
let rse = cb.makeResourceStateCommandEncoder()!
rse.updateTextureMapping(
tex,
mode: .map,
region: MTLRegionMake2D(0, 0, width, height),
mipLevel: 0,
slice: 0)
rse.update(fence) // ← capture all work so far
rse.endEncoding()
let ce = cb.makeComputeCommandEncoder()!
ce.waitForFence(fence)
ce.setComputePipelineState(pipeline)
ce.setTexture(tex, index: 0)
let threadsPerTG = MTLSize(width: 8, height: 8, depth: 1)
let tgCount = MTLSize(width: (width + 7) / 8,
height: (height + 7) / 8,
depth: 1)
ce.dispatchThreadgroups(tgCount, threadsPerThreadgroup: threadsPerTG)
ce.updateFence(fence)
ce.endEncoding()
// Blit texture into shared buffer
let blit = cb.makeBlitCommandEncoder()!
blit.waitForFence(fence)
blit.copy(
from: tex,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSize(width: width, height: height, depth: 1),
to: dstBuf,
destinationOffset: 0,
destinationBytesPerRow: rowStride,
destinationBytesPerImage: totalBytes)
blit.endEncoding()
cb.commit()
cb.waitUntilCompleted()
assert(cb.error == nil, "GPU error: \(String(describing: cb.error))")
// ── Verify a few texels
let out = dstBuf.contents().bindMemory(to: UInt32.self, capacity: width * height)
print("first three texels:", out[0], out[1], out[width]) // 0 1 64
assert(out[0] == 4 && out[1] == 4 && out[width] == 4)
}
Metal shader:
#include <metal_stdlib>
using namespace metal;
kernel void addOne(texture2d<uint, access::write> tex [[texture(0)]],
uint2 gid [[thread_position_in_grid]])
{
tex.write(4, gid);
}
Hello,
I'm currently working on my first SceneKit game and have encountered an issue related to moving an SCNNode using a UIPanGestureRecognizer.
When I deploy the game to my iPhone via Xcode in debug mode, all interactions are smooth. However, when I stop the debugging session and run the game directly from the device (outside of Xcode), the SCNNode movement behaves inconsistently — it works sometimes smoothly and sometimes not and the interaction becomes choppy. The SCNNode movement is controlled using a UIPanGestureRecognizer.
Do you have any ideas what might be causing the issue?
Hi
I've noticed one issue in Metal HUD, but I'm not sure if it is a bug in the Metal HUD or if there is a purpose for this behavior.
Metal HUD has an option to send the data to system log in raw format where the numbers are like
metal-HUD: ,,,,,...,
https://developer.apple.com/documentation/xcode/monitoring-your-metal-apps-graphics-performance/
If the HUD is displayed, it works just fine, but it seems that when the HUD is hidden (with shift-F9), it still send the data to system log, but the numbers are the same all the time and are not updated while is still being updated.
I would expect that it should log the data no matter if the HUD is displayed or not, this of course leads to incorrect FPS calculations
Here is an example of the system log entries when the HUD is not visible:
Topic:
Graphics & Games
SubTopic:
General
My experience has been that ModelEntity(named:in:) can be used to load a USD file with a simple structure consisting of entities and model entities, and, critically, it will flatten the entity hierarchy down to a single ModelEntity, presumably reducing the number of draw calls.
However, can anyone verify that the following is true?
If ModelEntity(named:in:) is used to load a USD file from a RealityKit content bundle, it may fail when the USD file contains more complex data, such as shader graph material definitions, or perhaps for some other reason. I am not sure.
AND the error that ModelEntity(named:in:) throws in this case is
Cannot load RealityKitContent entity: Failed to find resource with name "<name>" in bundle
which would literally suggest that the file does not exist, instead of what I assume the error actually is, which is "the file exists but its entity hierarchy could not be flattened to a single ModelEntity" ?
Is that an accurate description of the known behavior of ModelEntity:named:in:)?
I understand that I could use Entity(named:in:) instead, without the flattening feature. My question is really more about the seemingly misleading error message.
Thank you for any clarification you can provide.
I am trying to implement a ChacterControllerComponent using the following URL.
https://developer.apple.com/documentation/realitykit/charactercontrollercomponent
I have written sample code, but PhysicsSimulationEvents.WillSimulate is not executed and nothing happens.
import SwiftUI
import RealityKit
import RealityKitContent
struct ImmersiveView: View {
let gravity: SIMD3<Float> = [0, -50, 0]
let jumpSpeed: Float = 10
enum PlayerInput {
case none, jump
}
@State private var testCharacter: Entity = Entity()
@State private var myPlayerInput = PlayerInput.none
var body: some View {
RealityView { content in
// Add the initial RealityKit content
if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
content.add(immersiveContentEntity)
testCharacter = immersiveContentEntity.findEntity(named: "Capsule")!
testCharacter.components.set(CharacterControllerComponent())
let _ = content.subscribe(to: PhysicsSimulationEvents.WillSimulate.self, on: testCharacter) {
event in
print("subscribe run")
let deltaTime: Float = Float(event.deltaTime)
var velocity: SIMD3<Float> = .zero
var isOnGround: Bool = false
// RealityKit automatically adds `CharacterControllerStateComponent` after moving the character for the first time.
if let ccState = testCharacter.components[CharacterControllerStateComponent.self] {
velocity = ccState.velocity
isOnGround = ccState.isOnGround
}
if !isOnGround {
// Gravity is a force, so you need to accumulate it for each frame.
velocity += gravity * deltaTime
} else if myPlayerInput == .jump {
// Set the character's velocity directly to launch it in the air when the player jumps.
velocity.y = jumpSpeed
}
testCharacter.moveCharacter(by: velocity * deltaTime, deltaTime: deltaTime, relativeTo: nil) {
event in
print("playerEntity collided with \(event.hitEntity.name)")
}
}
}
}
}
}
The scene is loaded from RCP. It is simple, just a capsule on a pedestal.
Do I need a separate code to run testCharacter from this state?
I’m building an app that uses RealityKit and specifically ARConfiguration.FrameSemantics.personSegmentationWithDepth.
The goal is to insert an AR object into the scene behind a person, and an additional AR object in front of the person, while being as photo realistic as possible.
Through testing, I’ve noticed that many times, the edges of the person segmentation mask are not well matched to the actual person, and parts of the person are transparent, with the AR object bleeding through. It’s sort of like a “bad green screen” effect, which I’d expect to see a little bit, but not to this extent. I’ve been testing on iPhone 16, iPhone 14 Pro, iPad Pro 12.9 inch 6th Generation, and iPhone 12 Pro, with similar results across all devices.
I’m wondering what else I can do to improve this… either code changes, platform (like different iPhone models), or environment (like lighting, distance, etc).
Attaching some example screen grabs and a minimum reproducible code sample. Appreciate any insights!
import ARKit
import SwiftUI
import RealityKit
struct RealityViewContainer: UIViewRepresentable {
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
arView.environment.sceneUnderstanding.options.insert(.occlusion)
arView.renderOptions.insert(.disableMotionBlur)
arView.renderOptions.insert(.disableDepthOfField)
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = [.horizontal]
if ARWorldTrackingConfiguration.supportsFrameSemantics(.personSegmentationWithDepth) {
configuration.frameSemantics.insert(.personSegmentationWithDepth)
}
arView.session.run(configuration)
arView.session.delegate = context.coordinator
context.coordinator.arView = arView
}
func makeCoordinator() -> Coordinator {
Coordinator(self)
}
class Coordinator: NSObject, ARSessionDelegate {
var parent: RealityViewContainer
var floorAnchor: ARPlaneAnchor?
init(_ parent: RealityViewContainer) {
self.parent = parent
}
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
if let arView,floorAnchor == nil {
for anchor in anchors {
if let horizontalPlaneAnchor = anchor as? ARPlaneAnchor,
horizontalPlaneAnchor.alignment == .horizontal,
horizontalPlaneAnchor.transform.columns.3.y < arView.cameraTransform.translation.y { // filter out ceiling
floorAnchor = horizontalPlaneAnchor
let backgroundEntity = BackgroundEntity()
let anchorEntity = AnchorEntity(anchor: horizontalPlaneAnchor)
anchorEntity.addChild(background)
let foregroundEntity = ForegroundEntity()
backgroundEntity.addChild(foregroundEntity)
arView.scene.addAnchor(anchorEntity)
arView.installGestures([.rotation, .translation], for: backgroundEntity)
break // Stop after adding the first horizontal plane (floor)
}
}
}
}
}
}
Topic:
Graphics & Games
SubTopic:
RealityKit
I recently needed to develop an application to obtain the window list, which requires Screen Recording permissions. Apple's official documentation mentions using the two functions CGPreflightScreenCaptureAccess and CGRequestScreenCaptureAccess to request permissions. These functions are stated to be available since version 10.15. However, when I used these two functions on a device running macOS 10.15.7, I encountered the errors shown in the attached screenshot. I used the nm tool to inspect the symbols in the CoreGraphics.framework and found that these two functions were not present. Could you help me understand why this is happening?
Topic:
Graphics & Games
SubTopic:
General
Hello,
Im trying to install it following these steps https://www.applegamingwiki.com/wiki/Game_Porting_Toolkit but i get an error with 'brew install apple/apple/game-porting-toolkit'
==> tar -xf crossover-sources-22.1.1.tar.gz --include=sources/clang/* --strip-components=2
==> cmake -G Ninja -DCMAKE_VERBOSE_MAKEFILE=Off -DCMAKE_MAKE_PROGRAM=ninja -DCMAKE_VERBOSE_MAKEFILE=On -DCMAKE_OSX_ARCHITECTUR
Last 15 lines from /Users/rafael/Library/Logs/Homebrew/game-porting-toolkit-compiler/02.cmake:
-DLLVM_INSTALL_TOOLCHAIN_ONLY=On
-DLLVM_ENABLE_PROJECTS=clang
/private/tmp/game-porting-toolkit-compiler-20250519-44600-qwrjgl/llvm
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
-- Configuring incomplete, errors occurred!
If reporting this issue please do so to (not Homebrew/* repositories):
apple/apple
MacOS 15.3.1
Thank you in advanced
Regards
I am currently using RealityKit (perspective camera) to render a character in my swiftUI app.
The character has customization such as clothing items and hair and all objects are properly weighted to the rig.
The way the model is setup in Blender is like so: Groups of objects that will be swapped (ex: Shoes -> Shoes objects) and an armature. I then export it to usdc with all objects active. This is the resulting hierarchy:
Before exporting for the animation (armature modifier applied), I simply had to store the Model entities and swap them in but now when I export with the Armature Modifier applied, so that animations get exported, the ModelComponent gets flattened to the armature and swapping entities and applying new materials to them is no longer as simple.
Here's a demo blend file and usdc export with a setup like mine, having an animated bone to swing a cube and sphere, to be swapped so that only one is visible https://www.dropbox.com/scl/fo/be2q6qcztc83z7c4gj1w0/AMapxWc_ip2KZ8oTOYDUMv8?rlkey=rcdaggcxq06dyen09mw5mqmem&st=bnc0d7j0&dl=0
This is how I'm loading the entity and removing a part, with the demo files
import SwiftUI
import RealityKit
struct SwapDemoView: View {
var body: some View {
RealityView { content in
let camera = PerspectiveCamera()
camera.transform.translation = SIMD3(x: 0, y: 0.1, z: 3)
guard let root = try? await Entity(named: "simpleSwapDemo") else { fatalError("simpleSwapDemo.usdc is not present") }
print(root) // Get initial hierarchy
guard let cube = root.findEntity(named: "Cube") else { fatalError("Entity cube doesn't exist") }
cube.removeFromParent() // <-- Cube is still visible after removal
print(root) // Get hierarchy to confirm removal of cube
let resource = root.availableAnimations[0]
root.playAnimation(resource.repeat())
content.add(root)
content.add(camera)
}
.background(.white)
}
}
And this is what the entity hierarchy looks like in RealityKit before cube removal
▿ 'root' : Entity, children: 1
⟐ SynchronizationComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : ModelEntity, children: 2
⟐ SynchronizationComponent
⟐ ModelComponent
⟐ SkeletalPosesComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Primitives' : Entity, children: 2
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Cube' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Cube' : Entity
⟐ SynchronizationComponent
⟐ Transform
And here's the hierarchy after removal
▿ 'root' : Entity, children: 1
⟐ SynchronizationComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : ModelEntity, children: 2
⟐ SynchronizationComponent
⟐ ModelComponent
⟐ SkeletalPosesComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Primitives' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity
⟐ SynchronizationComponent
⟐ Transform
And this is the result:
What's the best practice here? Should animation be exported separately and then applied to the skeleton? If so, how is that achieved? I'm not really sure how to proceed here.
I am currently using RealityKit (perspective camera) to render a character in my swiftUI app.
The character has customization such as clothing items and hair and all objects are properly weighted to the rig.
The way the model is setup in Blender is like so: Groups of objects that will be swapped (ex: Shoes -> Shoes objects) and an armature. I then export it to usdc with all objects active. This is the resulting entity hierarchy, viewed in Reality Composer Pro:
My problem is that when I export with the Armature Modifier applied to the objects, so that animations get exported, the ModelComponent gets flattened to the armature and swapping entities is no longer as simple as removing the entity with the corresponding name.
What's the best practice here? Should animation be exported separately and then applied to the skeleton? If so, how is that achieved? I'm not really sure how to proceed here.
I'm trying to build an MDLMesh then add normals
let mdlMesh = MDLMesh.newBox(withDimensions: SIMD3<Float>(1, 1, 1),
segments: SIMD3<UInt32>(2, 2, 2),
geometryType: MDLGeometryType.triangles,
inwardNormals:false,
allocator: allocator)
mdlMesh.addNormals(withAttributeNamed: MDLVertexAttributeNormal, creaseThreshold: 0)
When I render the mesh, some normals are (0,0,0). I don't know if the problem is in the mesh, or in the conversion to MTKMesh. Is there a way to examine an MDLMesh with the geometry viewer?
When I look at the variable values for my mdlMesh I get this:
Not too useful. I don't know how to track down the normals.
What's the best way to find out where the normals getting broken?