damanloox Posted October 26, 2020 Report Share Posted October 26, 2020 (edited) Could somebody point me in the "right" direction regarding linking asm with VS .net? I've a simple function in asm as follows: global _addme _addme: push ebp mov ebp, esp ; add mov eax, [ebp+8] add eax, [ebp+12] mov esp, ebp pop ebp ret and compiling using nasm as follows: nasm -f obj -o addme.obj addme.asm Then (in my .cpp) extern "C" { int addme(int, int); }; and calling it val = addme(1,1) It is linked etc. when I build the project but it seems the function never returns (or doing something really wrong as the xbox just hangs after the call)... I don't have debug kit/environment setup (yet) so can't really do any meaningful debugging... Edited October 26, 2020 by damanloox Quote Link to comment Share on other sites More sharing options...
Arakula Posted October 27, 2020 Report Share Posted October 27, 2020 (edited) Depending on the calling convention, you might need to do a "ret 8". Or declare it as extern "C" { __cdecl int addme(int, int); } to be dead sure it doesn't use __stdcall or __fastcall, whatever the configured compiler options might be. And what you can always do: tell the compiler to emit an assembly output (/Fc, if memory serves me right) and check whether it emits "add sp,8" after the call to _addme. If not, your assembler code has to do it. Edited October 27, 2020 by Arakula Quote Link to comment Share on other sites More sharing options...
damanloox Posted October 27, 2020 Author Report Share Posted October 27, 2020 (edited) looks like the compiler does what it's supposed to do: (...) call _addme add esp, 8 (...) I have a feeling it has something to do with nasm (and perhaps generated output) and xbox... Pretty much the same code works on win32... On XBOX - it just hangs... Is the -f obj correct output? When I try using "bin" format (ie. without -f) I'm getting linker error (xbox_101 fatal error LNK1136: invalid or corrupt file)... So I thought obj is correct (didn't try win32 though - perhaps that's what I should use...)? Edited October 27, 2020 by damanloox Quote Link to comment Share on other sites More sharing options...
damanloox Posted October 27, 2020 Author Report Share Posted October 27, 2020 update... -f win32 did the trick 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.