Quick Links: Download Gideros Studio | Gideros Documentation | Gideros Development Center | Gideros community chat | DONATE
UrlLoader slow on Windows — Gideros Forum

UrlLoader slow on Windows

PaulHPaulH Member
edited December 2016 in General questions
I'm seeing a big difference in download speed using UrlLoader between platforms. Downloading a 15 MB file over the same internet connection with the same Gideros app, the Windows desktop version takes much longer than expected:

Android app: 20 seconds
App in Gideros Player on Windows: 20 seconds
WinRT app: 90 seconds

Any ideas why the Windows application would be over 4 times slower than either the player or Android version?

Could this be related to the issue where the WinRT implementation of UrlLoader doesn't trigger progress events except once at the start and once end of the download?

Paul

Comments

  • hgy29hgy29 Maintainer
    @PaulH, I admit that although it works, WinRT implementation of UrlLoader may not be really efficient: buffers are way too small, hence data is appended by small chunks into a growing buffer (realloc'ed each time a new chunk of bytes is received), etc.
    So I see reasons why it would be slow, indeed. And the way progress is supposed to work in WinRT is a mystery!
  • Thanks for the reply.

    Is this inherent in the WinRT environment, or is it possible to improve the implementation of UrlLoader in Gideros?

    Of course, I'm mostly glad it works at all. When I first tested an in-app download in the WinRT app I thought it wasn't downloading, between the lack of progress events and the much longer download time. I display a progress bar saying "Downloading..." during the process, and the green bar that should be filling it never appears. I made the whole bar flash slowly to prevent giving users the impression the app has locked up.

    Paul
  • Another observation on the WinRT download: If the app loses focus (the user switches to a different application while the download is running) the download makes no progress. I've had a user report half an hour waiting for a download to finish, one that takes me 1.5 minutes on WinRT, and 20 seconds on Android, and I suspect that moving the focus to another app is the reason that it didn't finish in 30 minutes.

    Is the WinRT code for downloads something I can look at directly? If I can help by making that more efficient, I'd be happy to do so for my own sake as well as that of the Gideros community.

    Paul
  • hgy29hgy29 Maintainer
    Hi @PaulH,

    You'll need to fetch gideros sources from github (https://github.com/gideros/gideros) and compile WinRT libraries yourself. Luckily, this is not the most complex part of gideros build. You'll find a batch file in scripts folder to get you started building WinRT libs: scripts/buildwinrt.bat. UrlLoader implemention will be in libgid/src/winrt/ghttp_winrt.cpp

    Thanks for you help :)
  • PaulHPaulH Member
    I'm finally working on this issue again as it's starting to impact customers significantly. In trying to build the WinRT libraries, whether from the scripts using msbuild or from Visual Studio, the build fails because dx11_shaders.c is not found. I can't seem to find this file anywhere. Is it missing from the github repository, or am I just overlooking something?

    Paul
  • hgy29hgy29 Maintainer
    This file is generated by a script, there is a compileshaders.bat somewhere to do that
  • PaulHPaulH Member
    Ah. That makes sense. Thanks.
  • PaulHPaulH Member
    I see two versions of compileshaders.bat: One in master\winrt, which tries to access texture.hlsl in that folder, which isn't there. The other compileshaders.bat is in mater\2dsg\gfxbackends\dx11, which I assume is the right one. But that one uses bin2, but I don't see an executable or script called bin2c. There's a bin2c.c, so I built an executable out of that, but its command line arguments don't match the way bin2c is called in compileshaders.bat. I'm definitely missing something here.

    Is there any documentation on setting up the build environment to compile the WinRT libraries? So far I've worked out that I need a command line path that includes Windows Kits 8.1 to pick up fxc.exe, and MSBuild to pick up msbuild.exe.

    Paul
  • hgy29hgy29 Maintainer
    Yeah, there are two ways to build gideros: the script based version (.bat/.sh) and the Makefile based one, and both versions don't always use the same tools. bin2c.c is used by the Makefile version, while .bat file used to use a prebuilt version from somewhere on the internet.
    Since you've built bin2c.exe from gideros sources, here is the relevant part of /scripts/GidWinRT.mk to build the missing file:
    WINRT_SHADERS=Basic Color Texture TextureAlpha TextureColor Particle Particles PathFillC PathStrokeC PathStrokeLC
    WINRT_SHADERS_PATH=2dsg/gfxbackends/dx11
    WINRT_SHADERS_FILE=dx11_shaders.c
    BIN2C=$(ROOT)/scripts/bin2c
     
    $(WINRT_SHADERS_PATH)/$(WINRT_SHADERS_FILE): $(BIN2C) $(addsuffix .hlsl,$(addprefix $(WINRT_SHADERS_PATH)/,$(WINRT_SHADERS)))
    	rm -f $(WINRT_SHADERS_PATH)/$(WINRT_SHADERS_FILE)
    	for s in $(WINRT_SHADERS); do $(FXC) //T vs_4_0_level_9_3 //E VShader //Fo $(WINRT_SHADERS_PATH)/v$$s.cso $(WINRT_SHADERS_PATH)/$$s.hlsl; done
    	for s in $(WINRT_SHADERS); do $(FXC) //T ps_4_0_level_9_3 //E PShader //Fo $(WINRT_SHADERS_PATH)/p$$s.cso $(WINRT_SHADERS_PATH)/$$s.hlsl; done
    	P=$(PWD); cd $(WINRT_SHADERS_PATH); for s in $(WINRT_SHADERS); do $$P/$(BIN2C) p$$s.cso >> $(WINRT_SHADERS_FILE); $$P/$(BIN2C) v$$s.cso >> $(WINRT_SHADERS_FILE); done
    	rm -f $(WINRT_SHADERS_PATH)/*.cso	
     
    winrt.shaders: $(WINRT_SHADERS_PATH)/$(WINRT_SHADERS_FILE)
    So basically just use your bin2c.exe with input file as single arg and redirect the output to the target file (with appending).
  • PaulHPaulH Member
    That helped. I wrote a batch job to do the equivalent steps using the bin2c.exe, so dx11_shaders.c seems fine now. The WinRT libraries build with lots of warnings for use of deprecated functions, but no errors. After swapping them into my WinRT exported project, the app crashes right after the launch screen, however. The debugger shows:

    ERROR FindDefaultInterface:CM_Get_Device_Interface_List_Size failed:0x00000000, InterfaceListSize = 1
    ERROR SensorOpenByType:Failed to open sensor error:0x00000490

    I can't find any relevant code. Any idea if this indicates a problem in the WinRT libraries I've built, or is there more that I'd need to bring over to my exported project besides the libraries themselves?
  • hgy29hgy29 Maintainer
    Those two lines aren't a problem, error is caught by code and ignored. Is there something else ?
  • PaulHPaulH Member
    Yes, after those errors, I see a "pad integer in precompiled chunk', then a few more libraries load, followed by some exceptions thrown:

    DataEntryScreen.lua: bad integer in precompiled chunk'giderosgame.Windows.exe'
    (... a few more libraries load ...)
    The thread 0x2d04 has exited with code 0 (0x0).
    Exception thrown at 0x761EA882 (KernelBase.dll) in giderosgame.Windows.exe: 0x40080201: WinRT originate error (parameters: 0x8000001A, 0x0000005D, 0x0361DAFC).
    Exception thrown at 0x761EA882 (KernelBase.dll) in giderosgame.Windows.exe: 0x40080201: WinRT originate error (parameters: 0x8000001A, 0x0000001D, 0x0361DCC0).
    The thread 0x1f44 has exited with code 0 (0x0).
    Exception thrown at 0x761EA882 (KernelBase.dll) in giderosgame.Windows.exe: 0x40080201: WinRT originate error (parameters: 0x8000000B, 0x00000040, 0x0381F6D0).
  • hgy29hgy29 Maintainer
    yes, that one is easy: it means that you are trying to use encrypted assets with an otherwise unencrypted gideros binary
  • hgy29hgy29 Maintainer
    btw, congratulation for compiling WinRT template :)
  • PaulHPaulH Member
    Interesting. So to use encrypted assets, I need Gideros to encrypt the library as it encrypts the assets during the export?
  • PaulHPaulH Member
    Does that mean if I replaced the libraries in (program files)\Gideros\Templates\VisualStudio\WinRT Template with the libraries I built, they'd get encrypted during the export, and that would prevent the exceptions I'm seeing?
  • hgy29hgy29 Maintainer
    Accepted Answer
    Yes, that's it
  • PaulHPaulH Member
    Cool. I just tried with non-encrypted source, and the app works. :) Just changing the buffer size from 1k to 100k sped up a download from almost 2 minutes to just over 15 seconds. I'll have a look at improving the efficiency of the code, but the buffer size change alone is a huge improvement.

    Thanks for all your help!
  • hgy29hgy29 Maintainer
    Thanks I'll enlarge buffer in official gideros code right now :)

    Likes: keszegh, PaulH

    +1 -1 (+2 / -0 )Share on Facebook
  • PaulHPaulH Member
    Sounds good. I tried with buffer sizes of 100KB and 1MB and the difference was minimal when downloading files around 15 MB, so from what I've seen the 100 KB seems a reasonable choice. Thanks again!

    Likes: antix

    +1 -1 (+1 / -0 )Share on Facebook
Sign In or Register to comment.