add altitudeAngle/azimuthAngle

[patrickhlauke]

Adapted the description slightly from https://w3c.github.io/touch-events/

Admittedly jotted down very handwavy advice note about implementations - this needs reviewing. Also, if this is taken up, we'll need relevant W3C tests for these new attributes.

Closes #274

---

Preview | Diff

[NavidZ]

Do you think it makes sense to include the transformation between these two sets of value as part of the spec?

[patrickhlauke]

@NavidZ @smaug---- et al, please have a look at the new informative section. i cribbed the core of the code from @k3a 's https://gist.github.com/k3a/2903719bb42b48c9198d20c2d6f73ac1 ... but my programmer chops are not what they used to be, so check that i didn't mess it up (also, opted to go for a simplified pseudo-code/javascript approach rather than Perl)

[patrickhlauke]

and in fact, I just realised that @k3a's code uses radians for both sets of values ... but tiltX/tiltY use degrees (between -90 and 90). so assume this needs extra conversion steps in both cases...?

[patrickhlauke]

also, looking at https://github.com/jquery/PEP/pull/376/files#diff-3b5aeb16d85e265fdb40c6063ac62a96R160 it seems there the cos and sin are flipped compared to this (if you look at the altitudeAngle/azimuthAngle to tiltX/tiltY conversion). which one's right?

[patrickhlauke]

the formulas are not reversible

that's probably because from what i can see, the altitude/azimuth approach has no concept of a pen's twist. if you tilt a pen from 0 to fully perpendicular and then carry on tilting further, in the tiltX/tiltY model you can tell the pen was moved over the perpendicular point, whereas in the other model it's the same as if you tilted the pen from the other side up. if that makes sense.

[NavidZ]

that's probably because from what i can see, the altitude/azimuth approach has no concept of a pen's twist. if you tilt a pen from 0 to fully perpendicular and then carry on tilting further, in the tiltX/tiltY model you can tell the pen was moved over the perpendicular point, whereas in the other model it's the same as if you tilted the pen from the other side up. if that makes sense.

Unfortunately I don't have a pen that reports a nonzero twist. But are you saying if I twist the pen the tilt axes will change and no longer parallel to the screen borders? I know we don't do any adjustment on Chrome and just report what we get from the OS.

[patrickhlauke]

no sorry, i mashed two thoughts together there. let me see if i can clarify: on windows etc. where tiltX/tiltY is used, pens also have a particular orientation/twist (e.g. where the button is on the barrel). and that lets you naturally distinguish between a pen that's only raised less than 90 degress from the surface (with the button pointing to the ceiling, if you will) and one that's gone "over" the perpendicular position (with the button pointing to the floor). but in the altitude/azimuth model, the second case doesn't exist. if you go over the perpendicular position, the azimuth simply goes 180 degrees the other way and the more you continue, the lower you're making the altitude again. you don't have a concept of a pen being upside down / gone past the perpendicular position (basically because altitude is only from 0 to 90 degrees, whereas in the tiltx/tilty model you have an additional "attitude" of the pen which makes you make more specific statements between 0 and 180 degrees).

anyway, sounds like this could get very complicated and edge-casey. something i hadn't considered initially.

also, worth going to the source of what @k3a based their code on, perhaps https://code.woboq.org/qt5/qtbase/src/plugins/platforms/windows/qwindowstabletsupport.cpp.html#527 as that seems to have some further nuances in the code?

[patrickhlauke]

thinking about this some more (though admittedly i'm slowly out of my technical depth here), essentially the two models differ at a fundamental level, tied to the kinds of sensors (or the data provided in the low-level/OS-level APIs). the tiltX/tiltY model is richer since it captures states that are simply not available/detectable in the altitudeAngle/azimuthAngle model ... those where a stylus is still pointing one particular way but the user went "over" the perpendicular mark.

wondering if it's sufficient to address this in prose? aknowledge that going from the tiltX/tiltY to the altitudeAngle/azimuthAngle model is "lossy"? and that on platforms that (natively/in their low-level APIs) only support the latter, certain nuances (like the pen being over the perpendicular) won't be captured/capturable?

also yes, @grorg @graouts i'd be interested if there's an official azimuthAngle for when a stylus is perfectly perpendicular (where it could be anything really)? is there a default/fallback of zero?

[NavidZ]

thinking about this some more (though admittedly i'm slowly out of my technical depth here), essentially the two models differ at a fundamental level, tied to the kinds of sensors (or the data provided in the low-level/OS-level APIs). the tiltX/tiltY model is richer since it captures states that are simply not available/detectable in the altitudeAngle/azimuthAngle model ... those where a stylus is still pointing one particular way but the user went "over" the perpendicular mark.

So looking at these pictures both of them show different sets of tiltX/tiltY and also altitude/azimuth values. Why do you say they have the same altitude/azimuth? They might have the same altitude as it seems they are in the same "height" (I'm just eyeballing it). But the azimuth is certainly different. For one on the left it is more close to 7xpi/4 and for the one on the right it seems to be 3xpi/4. I don't see the lossy model you are referring to here.

also yes, @grorg @graouts i'd be interested if there's an official azimuthAngle for when a stylus is perfectly perpendicular (where it could be anything really)? is there a default/fallback of zero?

Regarding the straight pen and the fact that perpendicular state could have any azimuth, we do have a similar situation for tiltX/tiltY. But at another location which is less likely to happen. That is when the pen is lying on the x axels on the surface. That can have any tiltY. Having said that I'm not saying we even get any events from the device in that case as the tip might not be even touching the screen and hence the device might not generate any events. So that is just hypothetical.

[patrickhlauke]

it might be that i'm getting myself confused here. need to dig out my pen and look at some values coming back. i somehow had it in my head that there's a distinction between these two states, but i might be completely wrong (and if i am, good)

[edit: the difference is probably only in the twist of the pen rather than tiltX/tiltY ... i think i got myself all confused thinking in theory, i should just open up and use my tests and echo out the raw values i think to get my mind straightened out]

[NavidZ]

it might be that i'm getting myself confused here. need to dig out my pen and look at some values coming back. i somehow had it in my head that there's a distinction between these two states, but i might be completely wrong (and if i am, good)

[edit: the difference is probably only in the twist of the pen rather than tiltX/tiltY ... i think i got myself all confused thinking in theory, i should just open up and use my tests and echo out the raw values i think to get my mind straightened out]

Yeah. I don't have a pen that supports twist in hand to see what its driver reports. But my understanding is that if you hold the pen at a particular position in space and just twist it only the twist value should change and not tiltX/tiltY. @Summerlw do you happen to have the stylus that reports twist handy to test this?

[liviutinta]

I did the trig math for altitude/azimuth so please have a look. Let me know if I made mistakes.

If pen's tip is resting at (x0, y0, 0) and let's say the length of the pen is L=sqrt(a^2+b^2+c^2) then the pen is spatially fully described by its tip (x0, y0, 0) and cap (a+x0, b+y0, c).

Now we can compute tiltX, tiltY, altitude and azimuth in terms of a,b,c. The terms x0,y0 do not matter as they are not participating in computing distances. x0, y0 will not appear going forward.

TiltX angle is described by the angle between normals to the planes (pen, Y) and (Z,Y): N(pen,Y) = (-c, 0, a) and N(Z,Y)=(-1, 0, 0). Which gives us:

cos(tiltX)=c/sqrt(a^2+c^2) 
sin(tiltX)=a/sqrt(a^2+c^2)
tan(tiltX)=a/c

TiltY angle is described by the angle between normals to the planes (pen, X) and (Z,X): N(pen,X)=(0,c,-b), N(Z,X)=(0,1,0). Which gives us:

cos(tiltY)=c/sqrt(b^2+c^2)
sin(tiltY)=b/sqrt(b^2+c^2)
tan(tiltY)=b/c

Altitude is the angle between the vector (a,b,c) and plane (X,Y) described by angle between vectors (a,b,c) and N(X,Y) = (0,0,1). Which gives us:

cos(altitude)=sqrt(a^2+b^2)/sqrt(a^2+b^2+c^2)
sin(altitude)=c/sqrt(a^2+b^2+c^2)
tan(altitude)=c/sqrt(a^2+b^2)

Azimuth is the angle between the projection of the pen on (X,Y) plane and the X axis. The projection of the pen has vector (a,b,0), the X axis has vector (1,0,0). Which gives us:

sin(azimuth)=b/sqrt(a^2+b^2)
cos(azimuth)=a/sqrt(a^2+b^2)
tan(azimuth)=b/a

Having the above formulas it is easy to verify the formulas at https://github.com/jquery/PEP/pull/376/files#diff-3b5aeb16d85e265fdb40c6063ac62a96R160 to be correct:

tiltX=Math.atan(Math.cos(azimuth) / Math.tan(altitude)) * radToDeg
tiltY=Math.atan(Math.sin(azimuth) / Math.tan(altitude)) * radToDeg

For computing altitude from tiltX and tiltY we can easily compute a and b in terms of c from tan(tiltY)=b/c and tan(tiltX)=a/c and then substitute in formula for altitude.

altitude=Math.atan(1/sqrt(Math.pow(Math.tan(tiltX),2)+Math.pow(Math.tan(tiltY),2))) 

For azimuth we can use function Math.atan2 where Math.atan2(y,x) represents the angle between X axis and vector (x,y,0).
Using tan(tiltY)=b/c and tan(tiltX)=a/c we have the following formula (making the assumption c > 0):

tanX = Math.tan(tiltX)
tanY = Math.tan(tiltY)
azimuth = Math.atan2(tanY,tanX)
if(azimuth < 0)
   azimuth += 2*Math.PI

Added code for the above here (code includes edge cases):
https://gist.github.com/liviutinta/0d040270ecf3390972421cb2164fbf37

[1] https://www.w3.org/TR/pointerevents3/
[2] https://w3c.github.io/touch-events/
[3] https://www.analyzemath.com/stepbystep_mathworksheets/vectors/vector3D_angle.html
[4] https://onlinemschool.com/math/library/analytic_geometry/plane_angl/

[patrickhlauke]

@liviutinta made a stab at changing the code to JavaScript (as I'm not sure if we want python code here - while in theory I guess any code would do, it feels a bit more natural providing it in JS here, as in the end it's just pseudo-code for browsers to then implement)). I'm not the cleanest or most elegant of coders, so happy for this to be made nicer/more modern/better documented.

[liviutinta]

After discussion with @NavidZ and while re-writing the Python code in Javascript, I was able to simplify the azimuth formula by using Math.atan2. I've updated my post above.
Javascript code is here: https://gist.github.com/liviutinta/0d040270ecf3390972421cb2164fbf37

[patrickhlauke]

thanks @liviutinta - i've rejigged it slightly (prefer to keep just two functions, rather than splitting out into helper functions for special cases). think it still all works as it should.

did the changes to the explanations/prose make (more) sense?

[liviutinta]

I have no problem understanding the definitions now. Really like the clock analogy as it makes it very clear what is going on.
For azimuth I believe it would be useful to specify that Y values increase in the direction of "6 o'clock" to mitigate any misunderstanding of the coordinates system.

[NavidZ]

I believe the default value of altitude should match the default value of tiltX/TiltY. So if they are 0 (for example when things aren't supported) then altitude should be pi/2.

One general suggestion is that if you add a new line after every comma or sentence it avoids the long lines here and easier to review. This is what I have seen in other specs as well and people seem to be generally happy with it.

[patrickhlauke]

regarding the default value: this matches what touch events level 2 says https://w3c.github.io/touch-events/#dom-touch-altitudeangle

i've also tested this on an ipad (with no stylus, just using finger) and a modified version of my HUD tracker (that explicitly still shows touch events, rather than pointer events) https://patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud-toucheventsonly.html and indeed azimuthAngle and altitudeAngle are both defined (so the props exist), and return 0. while i agree that it would be logical for the default to be pi/2, i think it's better to match touch events and what current implementations there do for defaults.

[patrickhlauke]

as for line length, agree...but as most other text in our spec already breaks this idea, i just went with this approach at this point. may need a good separate isolated reformatting pull request

[NavidZ]

I'm just trying to follow the same logic we did for pressure. In TouchEvent spec force is also zero if the device doesn't support it. But in PointerEvents it is 0.5 as we thought it makes more sense for the apps to still work with those devices.
With the same argument I believe we should diverge from TouchEvent spec in this case as well and set azimuth to pi/2 when it is not supported by the hardware. WDYT?

[patrickhlauke]

sorry, only saw this comment after i went ahead with the merge...

while yes, we diverged for the pressure, i think here the danger / potential for confusion is greater because we went for exactly the same property names as touch events. happy to revisit though if you/others feel strongly about it.

[patrickhlauke]

maybe altitudeAngle of 0 is simply assumed as being the default / not supported value because it's impossible (i assume) to really have that situation, since when the stylus is truly resting fully flat on the surface, the stylus tip isn't touching the screen itself so wouldn't be detectable anyway (unless you have a stylus that is "hover-capable" like, say, the pen on the galaxy note). i assume the apple pencil doesn't do hovering, in which case this assumption of "zero can't be the 'real' value" may make sense

[patrickhlauke]

confirmed (as i don't have a device myself) here https://twitter.com/yatil/status/1253658517241765895

[NavidZ]

I remember another reason for not exposing a "not supported" value was that developers didn't need to check for it every time. They just use the attribute and it works reasonably.
However, if we do go down the route of exposing a value that developers need to check for not supported case and do something other than what they would do with the actual value we are exposing I personally prefer to be much more explicit on that. Like expose an N/A or something that is clearly outside the range of possible values. I agree that Apple pen doesn't send 0 altitude at this point but I think we should be future proofing the API as well.

[patrickhlauke]

question is if even for non-apple-pencil stylus scenarios if there's a situation where a completely flat stylus would be detected or not.

i'm just thinking that IF there are web apps already dealing with altitudeAngle for touch events/pencil, they'd be familiar with the "if it's zero, just ignore this altogether" scenario. and because we use the exact same property name in PE, it may cause more confusion / need for forking code if in PE we then have different default.

either way yes, we need to document this more clearly either way, and also look at the pseudocode to check for that edge case/scenario.

[patrickhlauke]

split this out into #320

[NavidZ]

Beside the comment regarding the value of altitude needing to be pi/2 when it is not supported everything else looks good to me.

[patrickhlauke]

Not forgetting that we'll then also need matching w3c tests

[smaug----]

https://searchfox.org/mozilla-central/rev/05f486c76f84830041e51b7c68e3e697dc768454/testing/web-platform/tests/pointerevents/pointerevent_tiltX_tiltY_to_azimuth_altitude.html and others are testing this