Hot posts in thread: Alpha 10 performance issues beyond turn 340 on huge map

  1. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    Thanks a lot for the detailed report!

    I could not reproduce the issue here. I occupied Vrauc III, but everything went fine. However, your analysis and data (especially the output_log.txt you've attached) allowed us to spot where the problem may be. Basically, the code got stuck in an infinite loop (therefore the zeros you get in the log), which then caused the freeze and memory to fill-up.

    We don't know yet why you got this. Did you play only one game, or did you have the same issue on other games? If you played just this game, it can be specific to this game alone, for some reason we could not fully understand. When you load the Turn 366 game, and occupy Vrauc III, do you always get the freeze, or did the freeze happened just once and now you cannot reproduce this event when occupying Vrauc III again?

    We'll analyse this in more detail and come back to you when we find the solution to this problem. In the meantime, please keep reporting when you get these feezes and memory issues, and send us the output_log.txt top part if you can, so we can check if it is the same problem that is occurring, or something else.

    Thanks!
     
    Last edited: Jul 29, 2018
  2. Khan

    Khan Cadet

    Posts:
    20
    Joined:
    Jul 25, 2018
    Finally! Recreated problem. Turn 366/367, arrived at Vrauc III and decided to occupy enemy outpost. Application hung, memory ballooned.
    [​IMG]
    Drilled down for more detail ...
    [​IMG]
    The output_log.txt file has 3.7+ Gigabytes of "(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)" (not good).
    I also had the Performance Monitor working in the background (the trick being that once you set it, don't mess with it). I had it set to generate a CSV file and it caught all the params I had set up. I have attached the first 120+ kb of the log (you'll get the idea what the remaining 3.6 Gigs look like). In addition I zipped up the last two turns and the CSV I mentioned.

    Before I forget, you wanted to know what my page settings were ...
    [​IMG]
    Stupid question, Looking at the output_log.txt, could this be because I turned the music off?

    I hope this helps.
     

    Attached Files:

    Last edited: Jul 28, 2018
    • Helpful Helpful x 2
  3. Khan

    Khan Cadet

    Posts:
    20
    Joined:
    Jul 25, 2018
    Tried tonight to use perfmon to monitor ISG, big mistake. Very unstable and did not have the capability to monitor overall memory and CPU (very fine grain measurements, great if I knew what I was looking for). Anyways I had similar results to your's though the CPU did spike a bit. Hopefully I can find a tool that will give me a nice histogram of process specific usage. Turn 361.
     
  4. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    Thanks for the info Konstantine, that is really helpful.

    You are the person that has played the game the most so far. You're playing on a Windows 7 and don't report having memory issues or freezes of any kind, in large games even passing 350 turns. Seems like there's something specific going on here.

    It would not surprise me that an older version of Unity (the one we built PA10 with) could have issues with Windows 7. And, Windows 10 does have better memory management. But, it seems that something else may be involved, some specific option or system configuration. So far, no one else seemed to have complained about memory issues and freezes due to that. However, people tend not to play on Huge.

    Let's see what else Khan can give us on this issue.
     
  5. Konstantine

    Konstantine Grand Admiral

    Posts:
    2,200
    Joined:
    Oct 19, 2016
    Hi Both,

    I've been keeping up on this, this is what I can give you.

    While I hardly play huge in PA 10, what with three races only, I have played a few on large. I've built up some nice size empires and taken a few games past 350 turns, in one I approached 400 before getting bored.

    Some more info for you,

    I tend to have a lot of turns where I engage in an number of battles at the same exact time, (just my style;)) Apart from some issues that were bug related and reported in the past, I don't get any freeze up from this

    Now here is the kicker

    I got a new machine a while ago, I dumped windows 10 on purpose and installed 7 business, removed all bloatware that came with the machine as well. It's nothing special, just a Dell latitude with an I5 and 8 gb ram.

    So, taking the game to over 350 turns an a large map, never gave me an issue, though I don't recall ever researching Expert logistics,
     
    • Helpful Helpful x 1
  6. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    130 turns is too soon to see performance or memory related issues. If the freezes you report are due to memory running out than the problem must be local to Win7 or to some other issue in your machine. You do report having issues with other Unity games. Could it be the way memory is being handled? Could you check if you're allowing windows to control the paging file size for virtual memory, for example? It shouldn't be that because you have plenty of RAM, but still it's worth a shot.

    We had quite a few reports from people playing past 150 and 200 turns and no one reported having memory issues. I don't know which operating system they were playing on, could be Win10. We'll be asking people what version of Windows are they using from now on.

    Let us know how it went on your Win10 PC.

    Thanks.
     
    Last edited: Jul 27, 2018
  7. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    Could be. I noticed that Windows 10 does memory management much better than Windows 7 the moment I switched. I only have 6GB of RAM, and Windows 7 was becoming an issue. Now in Windows 10 I run everything fine with 6GB of RAM and the windows experience is really much quicker and smoother in Win10, no doubt about that.

    Theoretically, the game should run fine on Windows 7. There are a lot of recent and demanding Unity games that support Windows 7. Steam's HW stats report that around 32% of all PC users use Win7. Windows 10 users are now on 57%, says Steam.

    You report having issues with other games on your system, and your kids laptops as well. All have Win7. Let us know how it went in your Win10 PC. If the game has issues on Win7 we should have a look at that asap.

    Thanks!

    PS: By the way, the Chaos Chain weapon has a bug which may cause some enemy ships to become zombies. Just so you know if you experience that your combat stops at some point, it may be because of that. I've just checked that now.
     
  8. Khan

    Khan Cadet

    Posts:
    20
    Joined:
    Jul 25, 2018
    To clarify, the freeze first began taking place around turn 130! Freezes become more frequent as the game progressed. Now I'm beginning to wonder if there is some optimization Unity or something else the game uses on windows 10 vs 7. I've played around with Unity a little, as well as playing some other Unity based games. Every one of the Unity based games that is not a simple, top down, old style arcade shooter (think Super Mario or Asteroids) has experienced problems on my PC and my kids laptops (all have Windows 7) after running for a while. My youngest just got back from a game design workshop where they used Unity, his laptop has Win 7 (but I qualify that remark cause it's pretty borked and it desperately needs to be reloaded), his PC is Win 10 and I have him running through the problem now.

    The only straw I can grasp is that maybe an add-in has an unmet dependency not present in Win 7 (though the symptoms would not lend itself to such a conclusion)?
     
  9. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    No problem Khan, thanks for the save. To upload saves it's better to just zip them, as the uploader only accepts files with certain extensions, zip being one of them.

    I loaded the save, man, what a game you have going there! So many colonies, powerful fleets. Trading with your neighbors while they are at war. Nice.

    I played for 16 turns until Turn 355. Did some memory profiling and everything looks ok here. The game process started with around 2.5GB of RAM at the start (turn 339) but slowly dropped turn after turn until reaching around 1.1GB at turn 355. So, I see no memory issues on this side. I have Windows 10.

    That said, there is a known issue that causes a huge slowdown on the game in the late game. This major slowdown happens after researching the Expert Logistics tech, which gives unlimited supply range. This causes the Supply Range layer to perform quite badly which makes the game considerably slower. Try switching off the Supply Range layer (the icon with the parachute on the bottom-left) and you should see a considerable performance improvement.

    That could be causing you some of the issues, since you report the game being unplayable for you after turn 340. Expert Logistics will breakthrough in the game 8 turns later, so by turn 348 the game should become way less responsive.

    However, you also report freeze ups, having to force shutdown and having to reload the game from time to time after turn 340, which does suggest memory issues. But it's strange because you have a lot of RAM.

    Could the freezes be happening when you do combat (I got a freeze while playing the game during combat, and that was due to a bug)? Or if not, where exactly do you get the freezes, because these may be due to errors and bugs and not to memory issues.

    Do you notice the problem more during or after battles perhaps? I did not do any battles in my 16 turns until turn 355 but I did a couple battles in another game after playing until turn 345. I also did not notice any memory issues there but I did get a freeze, due to a bug.

    Could I ask you to check how much RAM does your "ISG_pre_alpha_10_64Bit" process takes while you play the game? Just to be sure if the problem is RAM, due to the supply range slowdown, or something else.

    Thanks a lot!

    ISG_process_RAM.jpg

    \Edit: By the way, here's the save zipped and in ready format after uncompressed to be loaded in case anyone wants to have a look.
     

    Attached Files:

    Last edited: Jul 27, 2018
  10. Khan

    Khan Cadet

    Posts:
    20
    Joined:
    Jul 25, 2018
    Hi Adam Solo,

    I apologize for the double post, it appears that outside of posts dealing with ISG, Space Sector is dead.

    P.S.: When I tried to upload the file (even shortened the name, no spaces, adding .sav to the end), but the site returns: "The uploaded file does not have an allowed extension. " I got around the limitation by adding the .txt suffix to it. Please note it is NOT a text file, it contains many non-printable characters.

    ------------------
    System Information
    ------------------
    Time of this report: 7/26/2018, 17:26:32
    Operating System: Windows 7 Ultimate 64-bit (6.1, Build 7601) Service Pack 1 (7601.win7sp1_ldr.180608-0600)
    Language: English (Regional Setting: English)
    System Manufacturer: MSI
    System Model: MS-7522
    BIOS: Default System BIOS
    Processor: Intel(R) Core(TM) i7 CPU 980 @ 3.33GHz (12 CPUs), ~3.3GHz
    Memory: 12288MB RAM
    Available OS Memory: 12280MB RAM
    Page File: 9211MB used, 15345MB available
    Windows Dir: C:\Windows
    DirectX Version: DirectX 11
    DX Setup Parameters: Not found
    User DPI Setting: Using System DPI
    System DPI Setting: 96 DPI (100 percent)
    DWM DPI Scaling: Disabled
    DxDiag Version: 6.01.7601.17514 64bit Unicode

    ---------------
    Display Devices
    ---------------
    Card name: NVIDIA GeForce GTX 760
    Manufacturer: NVIDIA
    Chip type: GeForce GTX 760
    DAC type: Integrated RAMDAC
    Device Key: Enum\PCI\VEN_10DE&DEV_1187&SUBSYS_847A1043&REV_A1
    Display Memory: 3767 MB
    Dedicated Memory: 1979 MB
    Shared Memory: 1787 MB
    Current Mode: 1920 x 1080 (32 bit) (60Hz)
    Monitor Name: ViewSonic VX2258wm
    Monitor Model: VX2258WM
    Monitor Id: VSC8E23
    Native Mode: 1920 x 1080(p) (60.000Hz)
    Output Type: DVI
    Driver Name: nvd3dumx.dll,nvwgf2umx.dll,nvwgf2umx.dll,nvd3dum,nvwgf2um,nvwgf2um
    Driver File Version: 24.21.0013.9836 (English)
    Driver Version: 24.21.13.9836
    DDI Version: 11
    Driver Model: WDDM 1.1
    Driver Attributes: Final Retail
    Driver Date/Size: 6/25/2018 13:26:02, 19083216 bytes
    WHQL Logo'd: Yes
    WHQL Date Stamp:
    Device Identifier: {D7B71E3E-52C7-11CF-9F55-77A41BC2D535}
    Vendor ID: 0x10DE
    Device ID: 0x1187
    SubSys ID: 0x847A1043
    Revision ID: 0x00A1
    Driver Strong Name: oem136.inf:NVIDIA_Devices.NTamd64.6.1:Section007:24.21.13.9836:pci\ven_10de&dev_1187
    Rank Of Driver: 00E62001


     

    Attached Files:

    • Helpful Helpful x 1
  11. Adam Solo

    Adam Solo Developer Administrator Grand Admiral

    Posts:
    4,847
    Joined:
    Sep 9, 2016
    As I replied in the SpaceSector blog comment, thanks a lot for your feedback, and welcome to the forums!

    It's great to see people playing huge games with so many turns. So far, most of the reports we got, and the games we played, were on Medium and Large, so it's good to have some feedback on Huge as well.

    As I said in the blog, we are aware that object pooling will probably be necessary everywhere due to how Unity manages objects and memory. We already do this in some places but not everywhere yet because the game seems to be running good in Medium and Large sized games with games around 200-250 turns, which allowed for people to play and provide feedback.

    We took note of this and we’ll make sure to address it during the optimization phase.

    And, by the way, since you played the game on Huge and for so many turns, let us know what other feedback you may have on the gameplay aspects, the good and the not so good as for any other feedback you can give us.

    Thanks!

    \Edit: By the way, could you please send us a save of that game so we can have a look? The saves are in: Drive:\Users\<user>\AppData\LocalLow\Praxis Games\ISG Please zip it and attach it to your post above or in a new post. Also, could you please let us know ho much RAM does your PC has? Thanks a lot!
     
    Last edited: Jul 26, 2018
  12. Khan

    Khan Cadet

    Posts:
    20
    Joined:
    Jul 25, 2018
    Please tell me you’re implementing object pools. Alpha 10 is unplayable beyond turn 340 on huge map (to get that far you have to reload the game every 8 -10 turns, where it will freeze up, requiring forced shutdown). Remember, you cannot explicitly destroy objects in Unity. The only way you can manage the bloat in Unity memory usage is by object pools reusing memory no longer needed.