I have a 3D endless runner game built in Unity3D running on iOS, Android and Windows Phone.
It works fine on Windows Phones with 1014MB memory and above, but under that it crashes as soon as it starts (see error message from Visual Studio at the bottom of this post).
Looking at the profiler I can see the memory usage for textures is extremely high, knowing that I only get a maximum of 180MB RAM on a Lumia 520 for example, I can see from the profiler that the textures take up 359MB alone, and even at eighth res (screenshot below) they still take up 232MB! I have gone through every texture making it as small and compressed as I can, am I doing something really wrong here? How can I find out more about what textures are problems?
I usually preload all the procedural level elements into memory at the start of the game to help the frame rate once you're running, but even when I turn this off completely the memory usage is unaffected.
Any pushes in the right direction would really help, thanks!
Visual Studio crash console log dump:
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: DefaultDomain): Loaded 'C:\windows\system32\mscorlib.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Windows.RuntimeHost.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Windows.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Net.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Xml.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\EndlessRunner.DLL'. Symbols loaded.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\Microsoft.Phone.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\Microsoft.Phone.Interop.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\WinRTBridge.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\BridgeInterface.winmd'. Module was built without symbols.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\UnityPlayer.winmd'. Module was built without symbols.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\WinMetadata\Windows.winmd'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Core.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\windows\system32\System.Runtime.ni.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Build from 'release/4.5/release' branch, version is '4.5.0f6 (fd4616464986)' (Release build).
Physical memory: 395 MB, commited memory limit: 180 MB.
PlayerConnection initialized from C:/Data/Programs/{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}/Install/Data (debug = 0)
PlayerConnection initialized network socket : 0.0.0.0 55444
Multi-casting "[IP] 192.168.0.19 [Port] 55444 [Flags] 2 [Guid] 93005236 [EditorId] 1016877170 [Version] 1048832 [Id] WP8Player(192.168.0.19) [Debug] 0" to [225.0.0.222:54997]...
Direct3D:
Version: Direct3D 11.0 [level 9.3]
Renderer: Qualcomm Adreno 305 (WDDM v1.2) (ID=0x30303330)
Vendor: Qualcomm
VRAM: 96 MB
Initialize engine version: 4.5.0f6 (fd4616464986)
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\UnityEngine.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\Assembly-CSharp-firstpass.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\Assembly-CSharp.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\BFSWP8Goodies.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\GPGSUtils.DLL'. Cannot find or open the PDB file.
'TaskHost.exe' (CLR C:\windows\system32\coreclr.dll: Silverlight AppDomain): Loaded 'C:\Data\Programs\{DFE540D3-80C1-4CFF-BFCA-CE252BC4EECE}\Install\WinRTLegacy.DLL'. Module was built without symbols.
Could not allocate memory: System out of memory!
Trying to allocate: 1398128B with 32 alignment. MemoryLabel: Texture
Allocation happend at: Line:411 in C:/BuildAgent/work/aeedb04a1292f85a/Runtime/Graphics/Texture2D.cpp
Memory overview
[ ALLOC_DEFAULT ] used: 5715133B | peak: 5715181B | reserved: 6570457B
[ ALLOC_GAMEOBJECT ] used: 43508B | peak: 43508B | reserved: 52731B
[ ALLOC_GFX ] used: 155628390B | peak: 155628390B | reserved: 155631656B
[ ALLOC_PROFILER ] used: 5028B | peak: 5028B | reserved: 12158B
Could not allocate memory: System out of memory!
Trying to allocate: 1398128B with 32 alignment. MemoryLabel: Texture
Allocation happend at: Line:411 in C:/BuildAgent/work/aeedb04a1292f85a/Runtime/Graphics/Texture2D.cpp
Memory overview
[ ALLOC_DEFAULT ] used: 5715133B | peak: 5715181B | reserved: 6570457B
[ ALLOC_GAMEOBJECT ] used: 43508B | peak: 43508B | reserved: 52731B
[ ALLOC_GFX ] used: 155628390B | peak: 155628390B | reserved: 155631656B
[ ALLOC_PROFILER ] used: 5028B | peak: 5028B | reserved: 12158B
(Filename: C:/BuildAgent/work/aeedb04a1292f85a/Runtime/Allocator/MemoryManager.cpp Line: 909)
The program '[2540] TaskHost.exe' has exited with code -2147483645 (0x80000003).
I have gone through every texture making it as small and compressed as I can, am I doing something really wrong here? How can I find out more about what textures are problems?
Hum.. I don't think you are doing something wrong. Memory limitations on hardware are hardware issues not artist issues. However, artist have to cope with restrictions in order to be able to cope with hardware limitations, then:
If you do 3D you can: make sure your models has as less materials as possible. The more materials you have on a model the more individual textures will most likely be required for such model. If you still need to, then make sure other models can share the same materials if possible or share some mesh groups so you can optimize on draw calls a tad.
If you can the usage of an Atlas texture is also good help, as its considered only one material. Then it will not help you that much with memory since it will still takes the same memory. It will only helps against draw calls mostly.
Like LearnCocos2D says, bits per color is a huge *if even if the textures are 'small' try not using 32 bit textures for example.
Texture resolution or texels are key. If you are using a character and your character is small at the mobile screen, then you should be good with really small textures. For example, in certain games the character is say 6' in height and an 8' foot wall is translated into a 512x512px texture in terms of coverage for say a brick's patch in a wall section that can look fairly good in contrast with the character. So for example going higher in such 8' foot equivalent height wall in contrast with such character will already be starting to waste resources and the visual impact will be minimum or not noticeable at all. It is important to define texel resolution for a 3D mobile game in order to be able to better plan your texturing resources.
I recommend you this tutorial if you can, its really informative about this issues: http://www.digitaltutors.com/tutorial/1772-Creating-Professional-Studio-Game-Assets-for-Production-in-3ds-Max-and-Unity
Then lastly in your case since its an infinite you really need to use enable/disable instead of destroy and also make sure the distance of the camera culling is exactly what you needed it be, otherwise you'll load lots of player unseen objects without need, this could be a boomer too... just saying.
I do see memory usage issues often in projects made with Unity.
Once you have gone through the typical optimizations such as texture compression settings, you can try another approach.
Since recently, Unity published a package called Addressables. This will allow you to keep references to your assets without them being loaded.
There, it is up to you when to load/unload them.
I wrote a detailed blog post about them, it might help you.
Find it in https://thegamedev.guru/unity-addressables-learn-workflows/
Related
I'm using RAD Studio 10.2.3 Tokyo.
Research
Related to this question, and this issue I run into trouble when debugging my flagship application that extensively utilizes runtime BPLs as a plugin system.
The problem occurs only when compiled and debugged as a Win64 application, and using Delphi's Win64 internal debugger. Delphi's Win32 internal debugger does not have this problem.
When using "load all symbols" in the debugger options, I miss stack and local symbols info for some units but not other units in the same BPL (check screenshots).
However, when I use the option to load debug info for specific BPLs, and only in my runtime BPL all units have local symbols info. Logically, I am missing debug info for all other units.
Of course, I dont like having to specify each symbol table specifically, I would rather load all symbol tables and not have to mess with it.
I have also experimented with using the symbol tables path, but this made no difference in loading the debugging results.
Question
Has anyone else experienced this, and found a better workaround than specifying which symbol tables to load?
Maybe something like an INI file, or Registry setting, or other solution for the Win64 debugger?
Some background info and screenshots below
Debugging view when turning on "Load all symbols"
Debugging view when using specific symbol table for this runtime BPL
Settings for specific symbol table loading
My Delphi win32 VCL Application is deployed as a single big exe. The application is a Client Server application with a Fat Client that connects to SQL Server.
This is convenient (it is enough to update one file in a shared folder and the application is updated- of course the DB schema gets updated too).
Since I use many "fat" third party components like ReportBuilder and DevExpress at every release the exe size grows mostly because those components become bigger and bigger.
In the dpr I set these flags to ensure when exe is launched from shared folder or from removable device it is always loaded in memory (and avoid odd connections errors):
{$SetPEFlags IMAGE_FILE_NET_RUN_FROM_SWAP}
{$SetPEFlags IMAGE_FILE_REMOVABLE_RUN_FROM_SWAP}
If the exe is built with the Release Build Configuration a normal exe usually reduces the size of about 50% (compared to Debug Build Configuration). But since I use EurekaLog, building with Build instead than Debug just slightly reduces the size.
Currently my exe is 115MB (built with "Build" Build configuration and EurekaLog).
Is it ok in anyone experience? Is it there a known limit that is advisable not to exceed?
Here's how the size has grown in recent years:
2014: 76MB
2015: 82MB
2016: 90MB
2017: 97MB
2018: 115MB
Since I recently exceeded the 100MB limit I started to worry.
I know that it is possible to build with runtime packages but my question focuses on keeping the single big exe approach if possible.
I remember cnPack has a Uses Cleaner feature I tried in the past. This could likely help me remove some non used unit and therefore reduce the exe size, but anyway this cannot stop the trend that brought me over 100MB.
Thanks.
I asked EurekaLog and they replied:
The size of your EXE after EurekaLog processing is mostly determined
by the size of your MAP file. Other things that affect the size of the
EXE include the EurekaLog options you select, such as memory checks,
Senders, JCL support, Etc.
You can reduce the size of the EXE somewhat by selecting the EL
Compression option. Keep in mind that compression can affect startup
time, since the debug information needs to be decompressed when you
launch the EXE.
You can further reduce the MAP file size by turning off debug
information in parts of your program that you don't need to stack
trace after a crash. Large component libraries like DevExpress would
be a good start.
Last, you may have other compiler options like range checking turned
on/off in tandem with EurekaLog. Some compiler options can contribute
to EXE size.
You can use our EurekaLog PE Analyzer to view details on debug
information size, Compression, Etc.
So somehow user Ville Krumlinde is right when he says that debug info is the cause. So to answer the question there are no problems in having a big exe, to reduce exe size the Release Build Configuration must be used, to use Eureka Log one should try to reduce he the map file size, in the above quote most hints are there.
You can use "Store all names externally" option to offload some debug information to external file, but your executables will no longer be self-contained.
I have a GUI application with size about 80MB (DEBUG mode). Sometimes when I change something in the source I just want to know if it compiles ok and I hit CTRL+F9 (I don't rely on Code Insight, because it doesn't always work on large projects). The compilation completes very fast, but the linking phase takes about 10 seconds on Core i7 with Samsung 840 Pro SSD!!! Which is pretty uncomfortable...
Is there an IDE option/shortcut to tell IDE to only compile the source, without linking/generating the actual .exe? Or DCC option?
I'm using Delphi XE7
Does FastMM makes your executables bigger? I notice my app grows about 300-500kb, which is rather big for me. I already turn off the delphi compiler option to exclude the debug info, but the size doesn't change.
I use the FastMM's FullDebugMode to on, is this affecting the size? But when I try to turn it off, it looks for libc, which I couldn't find.
Help please... I'm using D5
First of all, it doesn't really matter. Programs compiled with that option aren't meant for general distribution. They're meant to help you find the problem so you can fix it and then ship a correct version of your program without the debugging options enabled.
Second, half a megabyte is only big if you still distribute on floppy disks. Whether that's truly much space depends on how big your program was beforehand. Compress your program when you distribute it, and you'll probably hardly notice the difference.
Finally, to the crux of your question, yes, full debug mode makes your program bigger. It should be obvious that enabling that mode adds more code. It adds the code that does all the debug checking, such as checking for heap corruption, generating stack traces, catching the use of freed interfaces, logging errors to a file, and logging memory leaks. It also adds code to load and use the full-debug-mode DLL, and it changes any assembler functions to be written in plain Delphi instead.
The compiler option about debug information should have no effect. It controls whether debug information is included in the DCU files; that information is always excluded from the EXE files (unless you're talking about TD32 debug information, but you shouldn't enable that anyway since you're not using Turbo Debugger).
If your program is looking for the Libc unit, then you did more than turn of full debug mode. You somehow turned on Linux mode. You're not using Linux, so don't turn on that mode. If you're stuck, replace your copy of FastMM4Options.inc with the original version and start over.
The FullDebugMode use a pure pascal version of the FastMM4 heap manager.
This version marks blocks, so that any memory leak will be tracked and reported.
But in all cases, it won't NEVER add 300-500 KB to your executable.
It will be slower than "normal" FastMM4 mode, because of the pascal version of the coding (less optimized than asm) and because of the extra checks performed (e.g. the blocks marking).
But the 300-500 KB executable size increase is NOT related to FullDebugMode. It's only a matter of some dozen KB (in worst case), because most of the used code stays in the external FastMM_FullDebugMode.dll library.
You should have changed by hand some Project Options, like the TD32 info or the remote debugging info. But these info are not required with FastMM4 in full debug mode, as far as I understand.
FastMM should compile with D5 and it shouldn't look for libc, which is Linux library. There's problem with latest versions when compiling under D5 (like this), try the latest source code from source repository?
I'm using FastMM together with JCL Debug info to trace memory leaks in my application. However I have plugins which are dlls compiled in Delphi, both dlls and main application use common runtime packages. Now, when I'm shutting down the application, it generates memory leaks report in text file which is fine, but it contains stack traces only for main application. If memory leak occurs in plugin dll, the stack trace leads to LoadPlugins; procedure in main application!
Of course my plugins contain JCL debug data (it's inserted into .dll binary).
Where is the problem? Is that because of runtime packages usage?
Or some switch (define) which I forgot to enable/disable?
Thanks in advance
Have you set the FullDebugMode options and the Memory Manager Sharing Options (ShareMM and following) in FastMM4Options.inc?
Have you also compiled with debug infos?
Pierre, author of FastMM has answered my question. If anyone is interested, the solution is not to unload plugins at shutdown in order to keep stack traces & object types accessible. This however caused tons of bugs on shutdown (since FastMM was unloaded twice or so), so finally I had to move FastMM to shared package...