Gimbal lock fix?

[blue_max]

For anyone still interested in this, here's a preview of the gimbal lock fix:

[File removed, see the last message for an updated version]

The fix is activated by default, to toggle it, press Ctrl + Alt + G (you'll see a message confirming the change).

The gimbal lock fix only works in the cockpit at the moment, and I made an educated guess regarding the turn ratios and how they change according to throttle. There's still some work to be done here, but I think this is playable and people are welcome to try it out and report bugs.

[korekwerner]

Looks like the problem from 23 years ago is almost solved . What is missing in this ddraw are the shadows in the cockpit, but this is on the sidelines after the "gimbal lock" revolution. All made by XWA community wonderworker, master blue_max. Thank you.
Short test video skirmish - https://youtu.be/xKzZyI1ivOw

Gimbal lock fix seems to work in skirmish, however campaign mode and missions in general needs further hard work.
Behavior with gimbal fix enabled:
- strange behavior in the hangar (from what i can see, the change in the position of the player's ship (cockpit) is dependent on the calibration of the joystick and deadzone. This problem does not occur when I properly calibrate the joystick.)
Short video - https://youtu.be/BRRFRL1Zuzk

- incorrect placement of the player's ship after exiting hyperspace.
- Player ship rotated in hangar on mission restart.
short video - https://youtu.be/m65B8NHEA8Q

It seems as if the game is missing the reference point to which the player's ship is aligned.

[blue_max]

Looks like the problem from 23 years ago is almost solved . What is missing in this ddraw are the shadows in the cockpit

Thanks for testing this preview, Werner. Cockpit shadows should still work. Try setting the rendering mode to Deferred in Babu Frik, as I believe SSAO might be broken.

- strange behavior in the hangar

I'm aware of this problem, I just made a fix for that, but I haven't published the updated ddraw.

- incorrect placement of the player's ship after exiting hyperspace.
- Player ship rotated in hangar on mission restart.
short video - https://youtu.be/m65B8NHEA8Q

I didn't know about these, so thanks for reporting that. I'll fix it.

It seems as if the game is missing the reference point to which the player's ship is aligned.

I think what's happening is that the ship keeps the orientation it had before the mission is restarted -- but I'll check

[blue_max]

OK, here's an updated version that should fix the orientation problems in the hangar.


[File Removed download the latest version below]


The cockpit shadows are gone for some reason, I'm trying to figure out what happened.

[sedenion]

The gimbal lock fix only works in the cockpit at the moment, and I made an educated guess regarding the turn ratios and how they change according to throttle. There's still some work to be done here, but I think this is playable and people are welcome to try it out and report bugs.

It is secondary problem but It doesn't take joystickconfig.txt into account, so my axes setup is messed up, hard to properly compare.

Response to input is more precise than the original, but as I suspected, it is not the same flight model. The original flight model has inertia that is not "rendered" here.
I understand your enthusiasm to bypass the original flight model (and coding such thing is also passionating), but I don't share this enthusiasm because I don't think we are able to reproduce the original flight model... So, we will have a different game.

Anyway, do you have a public repository for your Ddraw like in github for example ? I would like to explore it, and even try by myself possibly.

[blue_max]

It is secondary problem but It doesn't take joystickconfig.txt into account, so my axes setup is messed up, hard to properly compare.

That file is read by Jeremy's joystick hook. It should continue to work as usual.

Anyway, do you have a public repository for your Ddraw like in github for example ? I would like to explore it, and even try by myself possibly.

It's all here:

https://github.com/Prof-Butts/xwa_ddraw ... c938bc6702

and the flight model can be customized by modifying the GimbalLockFix.cfg file. Feel free to propose different settings.

[blue_max]

OK, here's yet another version, I just fixed the cockpit shadows:


EDIT: File removed, download using the latest post below

[korekwerner]

Further tests with version 2 (version 3 behaves the same - except for cockpit shadows).
Problem with the direction of the player's ship - departure from the hangar and hyperspace.
Out of the hangar - random? arrangement of the ship's direction vector (X,Y,Z)
Hyperspace - direction vector correlated with the Y axis (north).
I hope this helps a bit.
short test video - easier to understand - https://youtu.be/29erN1O_Z9E

[sedenion]

That file is read by Jeremy's joystick hook. It should continue to work as usual.

This affect only axes assignation, but I now understand this is because I use two joysticks (stick + throttle). Your code work like there is only one joystick for all axes, while I actualy have pitch + yaw mapped on one and roll + throttle on the other.

[blue_max]

Problem with the direction of the player's ship - departure from the hangar and hyperspace.
short test video - easier to understand - https://youtu.be/29erN1O_Z9E

Thanks Werner. Yes, this helps me understand what's going on. I'll fix it.

[sedenion]

https://github.com/Prof-Butts/xwa_ddraw ... c938bc6702

and the flight model can be customized by modifying the GimbalLockFix.cfg file. Feel free to propose different settings.

Ok, I digged your code a bit and I have to admit I do not realy understand what you are doing. On reflection, I think I understood how the original flight engine work, I am pretty sure they use some acceleration model with damping mechanism, like a physics engine but without dealing with mass or such things.

So, instead adding rotation directly from input, you should increment and decrement a rotation speed variable (what you already do, but in a way and for a purpose I don't understand). The rotation speed is incremented by an arbitrary amount (depending on ship) and elapsed time. In the same time, this rotation speed must be damped according elsaped time at each frame.

I suggest you to try an implementation like the following one. This is pure pseudocode I wrote without testing, so, you'll have to adapt it. Once the damping factor and Flight Model values are properly "calibrated" the result should be closer to the original flight model, and more flexible.

Code: Select all

// Angular velocity variables, initialized to 0 at simulation start
float y_vel = 0.0f //< yaw angular velocity
float p_vel = 0.0f //< pitch angular velocity
float r_vel = 0.0f //< roll angular velocity

// Flight model acceleration constants. This describe amount of torque
// impulse to be applied accoding joystick input for each rotation axes. 

// In the example bellow yaw joystick input apply an impulse of 0.5 in 
// yaw and 0.2 in roll to the ship.
float fm_y[3] = {0.5f,  0.0f,  0.2f};
float fm_p[3] = {0.0f,  0.5f,  0.0f};
float fm_r[3] = {0.0f,  0.0f,  0.6f};

// Damping factor, must be adjusted to find proper value
float damp = 0.5f;

// simulation loop 
while(true) {
  
  // Troque-like acceleration "vector"
  float torque[3];
  
  // Compute vector according Flight Model values, assuming joy_x, joy_y
  // and joy_z are joystick X, Y and Z axes normalized input values.
  torque[0] = (joy_x * fm_y[0]) + (joy_y * fm_p[0]) + (joy_z * fm_r[0]);
  torque[1] = (joy_x * fm_y[1]) + (joy_y * fm_p[1]) + (joy_z * fm_r[1]);
  torque[2] = (joy_x * fm_y[2]) + (joy_y * fm_p[2]) + (joy_z * fm_r[2]);
  
  // Torque-like accumulation
  y_vel += torque[0] * frame_time;
  p_vel += torque[1] * frame_time;
  r_vel += torque[2] * frame_time;
  
  // Apply damping
  float d = powf(damp, frame_time);
  y_vel *= d;
  p_vel *= d;
  r_vel *= d;
  
  // Apply rotation to ship 
  ship_y += y_vel; //< ship yaw angle
  ship_p += p_vel; //< ship pitch angle
  ship_r += r_vel; //< ship roll angle

}

[blue_max]

// Angular velocity variables, initialized to 0 at simulation start
float y_vel = 0.0f //< yaw angular velocity
float p_vel = 0.0f //< pitch angular velocity
float r_vel = 0.0f //< roll angular velocity

This is what I called CurPlayerYaw/Pitch/Roll/RateDeg they are also initialized to 0.

// Flight model acceleration constants. This describe amount of torque
// impulse to be applied accoding joystick input for each rotation axes.

I called these g_fYaw/Pitch/Roll/AccelRate_s. They are also constants, but can be modified when the game starts because ddraw reads these values from GimbalLockFix.cfg

To explain how this works, I'm going to do an example for pitch.

Let's say the ship is flying straight and player pulls the stick all the way down. This is telling the system that the player wants to pitch as quickly as possible. So DesiredPitchRate_s is set to g_fMaxPitchRate_s (70 degrees per second, by default). Since the ship was flying straight, CurPlayerPitchRateDeg is 0, so now we need to do a smooth interpolation, over time, between CurPlayerPitchRateDeg (0) and DesiredPitchRate_s (70). Thus, at each time unit, you need to add the pitch acceleration to CurPlayerPitchRateDeg, but you need to compensate with the elapsed time to provide a consistent interpolation regardless of FPS. That's where the formulas:

const float DeltaPitch = DesiredPitchRate_s - CurPlayerPitchRateDeg;
...
CurPlayerPitchRateDeg += elapsedTime * g_fPitchAccelRate_s * DeltaPitch;

come from. g_fPitchAccelRate_s is 10.0, so every second, pitch will increase by 10 degrees. If your system is doing, say, 25fps, then this function gets called every 0.04s, so each time you want to add 10 * 0.04 = 0.4 degrees to CurPlayerPitchRateDeg. After 25 calls (1 second), you get 25 * 0.4 = 10 degrees per second. This is exactly what we wanted to achieve. Also, if your FPS isn't consistent, then elapsedTime will also adjust accordingly.

There are other details, like changing the max speed depending on the current craft's maneuverability and modulating the angular speed according to the throttle. I'm going to skip that for now, but it's all in the code too.

EDIT: corrected, damping should be applyed after rotation increment...

The dampening happens when you release the stick. In that case, CurPlayerPitchRateDeg will get interpolated from whatever value it currently has down to 0. I actually tried to add an extra dampening on every frame (after the current yaw/pitch/roll was incremented, exactly as you suggested), but that made the motion of the ship quite jerky -- especially when reversing directions. Also, rates and acceleration really hard to understand when dampening is on (because they compete with each other) and I ended up using nonsensical values (like 500 degrees per second) just to be able to control the ship.

I think that in order to get an experience that is guaranteed to be 100% the same, we would have to disassemble and reverse-engineer XWA's code. The relevant addresses of the functions are already in the comments in my github commit (see Direct3DDevice.cpp). But I think we can do a reasonable approximation by tweaking the values in GimbalLockFix.cfg. Just be careful when modifying this file: if you specify an acceleration rate that is too high, you run the risk of sending the system to NaN and you'll have to restart the game!

[sedenion]

The dampening happens when you release the stick. In that case, CurPlayerPitchRateDeg will get interpolated from whatever value it currently has down to 0. I actually tried to add an extra dampening on every frame (after the current yaw/pitch/roll was incremented, exactly as you suggested), but that made the motion of the ship quite jerky -- especially when reversing directions. Also, rates and acceleration really hard to understand when dampening is on (because they compete with each other) and I ended up using nonsensical values (like 500 degrees per second) just to be able to control the ship.

Ok, I now understand the issue. We need inertia but also constant turn-rate for a specific joystick input position, and this indeed cannot be done using torque logic. So what you coded can be understood as airplane control surface lag, and indeed if we down what you called "accel_rate" to 5, the flight model is more close to original.

Anyway, I noticed that this fix has another major drawback, it conflict with game original "physics engine", for example, when you are hit by a missile and the ship go in roll for a while... Noticing this I wonder if this inducted roll is of same nature of the one induced by joystick input.

[korekwerner]

I have no idea what you're talking about, but things are moving in the right direction. Sedenion is right, something is up with gimbalfix on after a missile hit. The ship's spins clip. When I disable gimbalfix in the same game session everything is ok.
here is short video - https://youtu.be/BmKidqjFOyE

[blue_max]

Alright, I've reduced the acceleration rates to 5 degrees per second, addressed the collision problems and I think I fixed the orientation of the ship after exiting hyperspace. Feel free to download the latest version here:

EDIT: File removed, see the last post for the latest version.

[korekwerner]

Great job blue_max. The fluidity with which the ship moves is extraordinary. Gaming on a gamepad is great, maybe the sensitivity is a bit too high, but the overall experience is very good. The direction of the player's ship has been fixed.
That's enough good news .
Enabling gimbalfix causes a change in the assignment of control devices (mentioned by Sedenion). Is it possible not to move assigned devices?
When we play on a gamepad everything is ok, but that's because we play on one device.
If we set more control devices and turn on gimbalfix then the game will replace the control devices. This happens in real time (which is great, but unfortunately it doesn't work properly yet).
Tests in progress....

[blue_max]

maybe the sensitivity is a bit too high

You can control the sensitivity through the yaw/pitch/roll acceleration rates in GimbalLockFix.cfg. If that doesn't work, let me know.

Enabling gimbalfix causes a change in the assignment of control devices (mentioned by Sedenion). Is it possible not to move assigned devices?

Did you assign the devices using the joystick hook? This fix is not yet 100% compatible with that hook. Try disabling that hook temporarily to see if the behavior changes.

[korekwerner]

Setting the sensitivity causes the ship to behave similarly to Squadrons, which is an interesting experience in XWA
After disabling the joystick hook, of course I can't play on three devices (joystick, throttle, rudder) . The gamepad works fine.
More tests and observations.
- after the first applause, unfortunately the departure from the hangar is random and the ship is randomly facing different directions.
- exit from hyperspace works fine.
- spins after missile impact need minor tweaking (there are noticeable minor cuts in the last second of spin.
test video - https://youtu.be/ocgniHPzCfg

[blue_max]

Setting the sensitivity causes the ship to behave similarly to Squadrons, which is an interesting experience in XWA

These settings are still a bit of guesswork. If you find better settings, let me know and I'll use them.

- after the first applause, unfortunately the departure from the hangar is random and the ship is randomly facing different directions.

Can you tell me if there are any standard XWA missions where this happens? I haven't noticed anything unusual, but I'll fix it.

Also, thanks for the extensive testing. This really helps me remove all the bugs faster.

[korekwerner]

I think I know how to recreate the hangar exit problem.
I checked on various game and hardware configurations (joystick, gamepads - Xbox and PS4).
The dependency is the calibration of the control device. It looks as if the game is constantly verifying the position of the gamepad/joystick sticks. At the first launch and calibrated devices, everything is ok. Multiple restarts of the same mission and forced non-center sticks cause unpredictable behavior when leaving the hangar (incorrect flight direction). Once we achieve this effect, it occurs in every mission. The problem does not occur when we jump into hyperspace.
Sometimes, exiting the game and re-entering does not reset gamepad/joystick settings.
I do reset these settings in a bit strange way, start the game/mission and press 5 on the numeric keypad (center TRACKIR key), I will add that I have TrackIR turned off during the tests. I then exit the game and restart it.
As I repeat this reset mode, I come to the conclusion that it does not work as I thought.
btw
I'm checking out other ships (except X-Wing) and there's more testing work to do. Strange behavior of cockpits eg YT-1300, very responsive, Y-Wing moved, I keep checking and see more fun. I will try to record various behaviors.
I hope I helped a bit.

[blue_max]

Thanks for helping me test this, Werner. Here's another version:

EDIT: File deleted, download the latest version below.

I have not addressed the problems in the hangar yet, but this version has a new, unofficial, version of the joystick hook. In other words, the gimbal lock fix should now be compatible with the joystick hook and now you can use multiple joysticks too.

Please before installing this version, rename your current "hook_joystick_ff.dll" to "hook_joystick_ff.dl_" to avoid conflicts between hooks.

The YT-series ships currently roll while doing yaw. I haven't figured out how to properly detect when a ship doesn't roll when doing yaw. I can inhibit roll by ship name, but that solution feels a bit clumsy to me. Other ships, like the Y-Wing should turn at a lower rate already, but perhaps the scale needs to be adjusted.

[korekwerner]

New joystickhook and ddraw after very short tests with many devices connected, I can confirm that switching between gimbalfix on/off modes works. It does not disconnect and change the axis/buttons of the devices. In addition, FF (Force Feedback - for a younger audience) works in both modes.
Tests in progress....

Thank you.

[batuta]

Thank you , what a great update.

[blue_max]

Thanks for helping me test this, guys. Here's a minor update. It contains better support for the mouse and there's a new setting that disables "throttle modulation" (the thing that makes your ship turn faster at 1/3 throttle). Apparently, classic TFTC doesn't have that feature, so it can be turned off for the Gimbal Lock fix now.


EDIT: File deleted, please download the latest official beta from the Downloads section in this forum

[keiranhalcyon7]

Apparently, classic TFTC doesn't have that feature, so it can be turned off for the Gimbal Lock fix now.

Understandable, since that feature didn't exist until at least XvT... but I wonder, how did TFTC disable it???