PREVIEW: Effects ddraw 2.0 with enhanced GPU support

All the XWAU recent release news from the project can be found here
Post Reply

Re: PREVIEW: Effects ddraw 2.0 with enhanced GPU support

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Thu Jan 27, 2022 1:21 am

Ace Antilles wrote:
Wed Jan 26, 2022 10:38 pm
We're also hearing of little issues with different hangars (in TFTFC)
I'm trying to fix this. A quick workaround for this problem is to rename the hangar ".mat" files to ".ma_"
For instance, change "Materials\Platform2Hangar.mat" to "Materials\Platform2Hangar.ma_"
This isn't an actual fix, but it should allow you guys to fly missions while I work on this problem.

PunkFriday
Cadet 3rd Class
Posts: 25
Joined: Sun Nov 02, 2014 4:02 pm

Post by PunkFriday » Thu Jan 27, 2022 2:10 am

Will this be included in the next release? I'm too lazy for dropbox, lol

User avatar
AngeI
Ensign
Posts: 443
Joined: Sun Jan 24, 2016 5:27 pm

Post by AngeI » Thu Jan 27, 2022 4:59 am

PunkFriday wrote:
Thu Jan 27, 2022 2:10 am
Will this be included in the next release? I'm too lazy for dropbox, lol
Probably. But what's so lazy about clicking 'download' on a dropbox link? Its literally as easy as downloading from XWAU website :P

Muahaha
Cadet 4th Class
Posts: 19
Joined: Thu May 13, 2021 4:45 am

Post by Muahaha » Thu Jan 27, 2022 5:33 am

tested it on quick skirmishes, seems smooth with multiple capital ships and asteroids thrown in. Some color inconsistencies on the ISD but no artifacts seen overall.
I wonder with this hook, how big of a battle it can be, can't wait.

Thanks

McMyn
Cadet 4th Class
Posts: 12
Joined: Thu Jan 27, 2022 11:11 am

Post by McMyn » Thu Jan 27, 2022 11:22 am

First of all, thanks for all of the amazing work. And this sounds like a really cool piece of progress for many people's performance!

On the subject of lack of VR support: is it clear how much of an obstacle there is? Like, does VR somehow have to be re-built from the ground up (many months or even years), or is it a question of figuring out a small number of quirks and then it should work (where it could be a matter of weeks).

Side question: I've looked at my performance data in VR yesterday, with the current "release" patch IMO (Update 5 on Mega Patch). There it looks like I'm GPU bound: 70-100% of GPU usage at 40-80 FPS, with way less than the 25% CPU use that I believe XWA can currently get on my CPU (I believe my numbers were 10-14% CPU use).
(Ryzen 5 5600, Geforce 1080).
For someone like me, is this update still worth pursuing? Or am I truly GPU bound and outsourcing geometry to the GPU as well would just worsen performance?

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 6603
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Thu Jan 27, 2022 1:12 pm

PunkFriday wrote:
Thu Jan 27, 2022 2:10 am
Will this be included in the next release? I'm too lazy for dropbox, lol
It will be when it's fully fixed, but that's quite a long way off right now
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

brassman2468
Cadet 4th Class
Posts: 17
Joined: Tue Jan 08, 2019 1:57 am

Post by brassman2468 » Thu Jan 27, 2022 1:40 pm

The Dropbox link appears to have been deleted, what happened?

User avatar
m0rgg
XWAU Member
Posts: 156
Joined: Wed Apr 01, 2020 10:33 pm

Post by m0rgg » Thu Jan 27, 2022 5:32 pm

McMyn wrote:
Thu Jan 27, 2022 11:22 am
First of all, thanks for all of the amazing work. And this sounds like a really cool piece of progress for many people's performance!

On the subject of lack of VR support: is it clear how much of an obstacle there is? Like, does VR somehow have to be re-built from the ground up (many months or even years), or is it a question of figuring out a small number of quirks and then it should work (where it could be a matter of weeks).
Definitely not a full rewrite, but it requires some adaptation. Once @blue_max gets to it (he may need to take some rest) it should be weeks, not months, at least for his genius.
Side question: I've looked at my performance data in VR yesterday, with the current "release" patch IMO (Update 5 on Mega Patch). There it looks like I'm GPU bound: 70-100% of GPU usage at 40-80 FPS, with way less than the 25% CPU use that I believe XWA can currently get on my CPU (I believe my numbers were 10-14% CPU use).
(Ryzen 5 5600, Geforce 1080).
For someone like me, is this update still worth pursuing? Or am I truly GPU bound and outsourcing geometry to the GPU as well would just worsen performance?
XWA is basically single threaded. The main game thread runs in a single core, which in the case of your CPU corresponds to 100%/6 cores = 16%, so your CPU is more or less maxed out.
The GPU is much better designed to do the kind of massively parallel work of geometry transformation.

It's not trivial to evaluate the bottlenecks, you can use some specific tools like GPUView or Intel GPA to analyze it. Also it may be completely different depending on the scene.

XWA suffers when there is a lot of geometry (many polygons), and that is limited by the CPU because that's where it is computed.
With the new hook, this needs to be re-evaluated to find the new bottleneck in the lowest performing scenes. There may be even additional opportunities for optimization on GPU work or CPU/GPU sync once the previous CPU bottleneck has been removed.

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Thu Jan 27, 2022 5:34 pm

brassman2468 wrote:
Thu Jan 27, 2022 1:40 pm
The Dropbox link appears to have been deleted, what happened?
Sorry about that. I just updated the DropBox link. Please download the newest ddraw, it has a few more fixes.

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Thu Jan 27, 2022 5:38 pm

@McMyn Like m0rgg said, it's a bit hard to predict, but I do expect there will be benefits in VR from this new ddraw version. At the very least, the drops in performance should be less frequent and less severe. It should be about a couple of weeks to port the VR version.

McMyn
Cadet 4th Class
Posts: 12
Joined: Thu Jan 27, 2022 11:11 am

Post by McMyn » Thu Jan 27, 2022 8:25 pm

So yeah, sorry everyone - I was talking a bit out of my own... exhaust port there.

I looked at my tool again just now, and fiddled with VR resolutions. Turns out I did not quite realize what "geometry is computed on CPU (rather than GPU as is typical now)" has as its implications. And that I cannot read frame times properly.

I deduced that I was GPU bound partly because my performance scaled so hard with resolution. But I guess that's mostly because the way this game's engine works, CPU use scales harder with resolution. At low resolution (like 20% of my Rift S' native res), I could pretty clearly see what was actually happening:
- my GPU frametimes were continually really low; like, negligible really, enough for hundreds of FPS
- my CPU frametimes were going up and down, and when they were up, they were definitely going above 12.5ms (Rift S has 80hz), so I went into lower frame territory
- at high CPU frame times (and this is what I did not understand at all before), my GPU had to work a LOT harder to finish the frame in time, because so much of the relevant rendering time was taken up by the CPU side; this is why my GPU was running at up to 98% capacity I think now, because its frametimes stayed below 1ms at all times IMO

Thanks for the hint about my CPU as well. Of course, it has 6 physical cores, so 16% is full use of one of them.

And that is such good news about VR, it's really all I care about performance-wise :) Like, I will play pancake mode with my kids, but I will also not mind the occasional frame drop so much there.

klickhom
Cadet 4th Class
Posts: 19
Joined: Sun Oct 25, 2020 3:13 pm

Post by klickhom » Thu Jan 27, 2022 8:54 pm

Tech Globe report:

Some objects have geometry errors... sometimes.
Closing and starting X-WA may fix some or break others or something.
Conclusion:
Models may be fine, but they aren't loaded properly or something. After skipping some they star looking broken randomly.

Videos there:
https://nrbkserver.duckdns.org/nextclou ... aA2bS6asJP

Played B7-M1 to 3

Looked in Tech globe and those things had geometry or texture errors:
BFF-1 Bulk freighter - Geometry
EF76 Nebulon-B Escort Frigate - Tail texture
XQ3 Platform (PLT/3) - Trasnparent walls
Probe 1 - Engine Glow
Rendezvous Buoy (R/BUOY) - inner side "empty"
TYpe 1 Navigation Buoy (NAV/B) - Botched
Z-95-AF4 Headhunter - Transparent + botched
Delta-class JV-7 Escort Shuttle (E/S) - Transparent + botched
YT-1300 Corellian Transport - Missing or transparent textures
YT-2000 Corellian Transport (YT-200)- Transparent + botched
Class-K Cargo Container - Botched
Class-L Cargo Container - Botched

Close XWing
Open XWIng
And some where fixed and others no...

ben1
Cadet 4th Class
Posts: 24
Joined: Sat Oct 17, 2020 4:35 pm

Post by ben1 » Fri Jan 28, 2022 12:49 am

I think it’s a great achievement what has been done here.

One thing which is a real shame is that I believe there are only 2 lods for each craft. If there were more lods then at a distance a craft could potentially be as little as 6 polys (for example). I have worked on racing games where there are 4 lods per car, the main A lod has 100,000 polys, which dwindles down all the way to lod D where it can be as low as 500 polys. This makes a huge performance difference with a full grid of race cars.

I guess there is no way to increase the number of lods per craft in the xwa engine?

Dax
Cadet 4th Class
Posts: 15
Joined: Sat Apr 24, 2021 4:26 pm

Post by Dax » Fri Jan 28, 2022 2:45 am

Hi all. Entered the new code as directed (ddraw.cfg and hook.ini). Replaced the old ddraw.dll. When I opened the game, I got a double screen. Two very small boxes, identical, of the concourse. When I moved to exit (ESC, click exit to Windows), the same actions happened on both little screens. What can I do to fix?

User avatar
Forceflow
Fleet Admiral (Administrator)
Posts: 6787
Joined: Wed Oct 20, 1999 11:01 pm
Contact:

Post by Forceflow » Fri Jan 28, 2022 6:54 am

ben1 wrote:
Fri Jan 28, 2022 12:49 am
I think it’s a great achievement what has been done here.

One thing which is a real shame is that I believe there are only 2 lods for each craft. If there were more lods then at a distance a craft could potentially be as little as 6 polys (for example). I have worked on racing games where there are 4 lods per car, the main A lod has 100,000 polys, which dwindles down all the way to lod D where it can be as low as 500 polys. This makes a huge performance difference with a full grid of race cars.

I guess there is no way to increase the number of lods per craft in the xwa engine?
Crafts can (and some already do) have multiple levels of details, the way this is done is by having smaller details simply vanish at distances where you cannot see them.
There are also ways to do multiple full LODs per craft but in all honesty this is rarely useful. Except for the largest of crafts details vanish so fast that a single low detail model is plenty. For the very large ships that can be implemented, but this is a lot of additional work, and with the limited number of people actively working on modelling it's just not something that can be done right now. There are also some engine limitations that make doing LODs a lot more difficult than it can seem at first.
Murphy was an optimist! I am a pessimist!
And always remember that a smile is cheaper than a bullet! (District 9)
Webmaster of the X-Wing Alliance Upgrade Project

brundlemouche
Recruit
Posts: 2
Joined: Fri Jan 28, 2022 1:49 pm

Post by brundlemouche » Fri Jan 28, 2022 1:51 pm

Hi
wanted to give it a try, but test dll seems removed from dropbox for a reason. Any new link to share ?

Thanks a lot !

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 6603
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Fri Jan 28, 2022 2:01 pm

brundlemouche wrote:
Fri Jan 28, 2022 1:51 pm
Hi. wanted to give it a try, but test dll seems removed from dropbox for a reason. Any new link to share ?
Thanks a lot !
Works for me. You won't get a preview. Just click download.
https://www.dropbox.com/s/5u5fyrimptj4x ... w.dll?dl=0
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

brundlemouche
Recruit
Posts: 2
Joined: Fri Jan 28, 2022 1:49 pm

Post by brundlemouche » Fri Jan 28, 2022 2:06 pm

Thanks mate, this one is working.

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Jan 28, 2022 6:04 pm

Dax wrote:
Fri Jan 28, 2022 2:45 am
Hi all. Entered the new code as directed (ddraw.cfg and hook.ini). Replaced the old ddraw.dll. When I opened the game, I got a double screen. Two very small boxes, identical, of the concourse. When I moved to exit (ESC, click exit to Windows), the same actions happened on both little screens. What can I do to fix?
Are you trying to run the game in VR? This ddraw version does not support VR yet. To restore your game, just revert the changes you did to ddraw.cfg and hooks.ini; and use the ddraw.dll from the "Effects Beta" link in the Downloads page in this forum.

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Jan 28, 2022 10:23 pm

klickhom wrote:
Thu Jan 27, 2022 8:54 pm
Tech Globe report:

Some objects have geometry errors... sometimes.
Thank you for reporting this. At this time I have just started implementing the VR path, so it's probably going to take a few more days before I come back to address this problem.

Bman
Lieutenant Commander
Posts: 1082
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Fri Jan 28, 2022 10:52 pm

I haven't checked, but for the platform .mat files or even other objects, I'm wondering if the texture files that are referenced are not mapped correctly. Spellings etc?
W-I-P: ISD-II, (Imp) Escort Carrier, NL-1 Platform, Misc, & TFTC.

User avatar
blue_max
XWAU Member
Posts: 2141
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sat Jan 29, 2022 1:42 am

Bman wrote:
Fri Jan 28, 2022 10:52 pm
I haven't checked, but for the platform .mat files or even other objects, I'm wondering if the texture files that are referenced are not mapped correctly. Spellings etc?
I also thought that might be the case, but no: everything works fine the old ddraw version. The problem was related to transparency. In this version, I have to manually defer all draw calls with transparent textures and play them near the end of each frame so that they display and blend correctly. In doing so, I made a mistake when storing the data. This platform just happened to have a transparent texture with additional properties (the force field) that exposed this problem. This was also related to the Tech Room crash. If the Tech Room was displaying a ship with a cockpit, the glass in the cockpit was also being deferred to the end of the frame because it has transparency. However, I forgot to properly consume this these draw calls because the Tech Room is a special case.

Since others have reported additional problems with the Tech Room, this suggest that my fix was incomplete and I need to go back. But since it's not interfering with regular gameplay, I consider it lower priority as I would like to enable VR first ;)

McMyn
Cadet 4th Class
Posts: 12
Joined: Thu Jan 27, 2022 11:11 am

Post by McMyn » Sat Jan 29, 2022 7:58 am

blue_max wrote:
Fri Jan 28, 2022 10:23 pm

Thank you for reporting this. At this time I have just started implementing the VR path, so it's probably going to take a few more days before I come back to address this problem.
If there is anything I could do (testing-wise or so) to support your VR efforts, especially but not only over the weekend, just let me know.

Dax
Cadet 4th Class
Posts: 15
Joined: Sat Apr 24, 2021 4:26 pm

Post by Dax » Sat Jan 29, 2022 3:27 pm

blue_max wrote:
Fri Jan 28, 2022 6:04 pm
Dax wrote:
Fri Jan 28, 2022 2:45 am
Hi all. Entered the new code as directed (ddraw.cfg and hook.ini). Replaced the old ddraw.dll. When I opened the game, I got a double screen. Two very small boxes, identical, of the concourse. When I moved to exit (ESC, click exit to Windows), the same actions happened on both little screens. What can I do to fix?
Are you trying to run the game in VR? This ddraw version does not support VR yet. To restore your game, just revert the changes you did to ddraw.cfg and hooks.ini; and use the ddraw.dll from the "Effects Beta" link in the Downloads page in this forum.
I am not using VR and don't have that capability. Was hoping there might be a workaround I'm not aware of that would allow me to try the new ddraw, which is a super exciting development! But I can also wait if need be.

User avatar
RandomChain
Cadet 4th Class
Posts: 17
Joined: Mon Nov 23, 2020 10:55 pm

Post by RandomChain » Sat Jan 29, 2022 8:42 pm

Have anyone got this running on Linux?
I'm using lutris-wine 5.6-5. Once it gets into the flight engine the game crashes with "DxvkMemoryAllocator: Mapping memory failed with VK_ERROR_MEMORY_MAP_FAILED" (not sure if that's really the relevant error).
I can get it to run with setting IsHookD3DEnabled = 1 but then no textures are loaded.

Post Reply