Skip Menu |

This queue is for tickets about the Win32-API CPAN distribution.

Report information
The Basics
Id: 42218
Status: resolved
Worked: 2 hours (120 min)
Priority: 0/
Queue: Win32-API

People
Owner: cosimo [...] cpan.org
Requestors: jim.brackett [...] milliman.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.57
Fixed in: (no value)



Subject: Win32::API::Callback Heap Allocation Size
Hello, While using Win32::API::Callback with ActiveState Perl, I observed corruption of a heap block which I think is attributable to an overwrite by one byte of the dynamically-allocated code buffer. Specifically, the high byte of the number of bytes to pop off the stack upon return from a stdcall (in my case, and presumably most everyone else's case, zero) is placed at offset tomalloc, which -- given the first byte is at offset zero -- exceeds the allocation size by one byte, resulting in heap corruption. I have not had an opportunity to recompile the extension for debugging, but I believe the error may be the addition of 3 bytes instead of 4 to account for stdcall epilogue; I note at the bottom of CallbackCreate, you deduct 2 bytes from the cursor, then proceed to emit 6 bytes of machine code for the epilogue, which -- if I understand the other pointer arithmetic correctly -- results in a net addition of 4 bytes, not 3. Although I did not recompile from source, I did apply a binary patch to the DLL to replace the +3 with +4, and the modified binary worked perfectly, precisely filling the heap block with code, and not yielding any heap corruption. I hope this information proves helpful. Thanks for such a great job on a very valuable Perl extension! Kind regards, Jim
Jim wrote: Show quoted text
> Although I did not recompile from source, I did apply a binary > patch to the DLL to replace the +3 with +4, and the modified binary > worked perfectly, precisely filling the heap block with code, and not > yielding any heap corruption.
Hi Jim, I made the change, but I'd like to add a proper test case for that. Can you send me the example script that caused the problem? Or can you help me build a test case for this bug? Thanks again.
Subject: Re: [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size
Date: Tue, 13 Jan 2009 15:13:53 -0600
To: bug-Win32-API [...] rt.cpan.org
From: Jim Brackett <jim.brackett [...] milliman.com>
The original offending script is a bit on the heavy side, but I'm sure I can come up with something that trips the overwrite. The trick might be in systematically identifying the problem when it happens, since in my case it was detected during a debugging session, based in part on compiler specifics; I think the heap from which the memory is allocated depends on the distribution of Perl, so I'm not sure how best to interrogate heap integrity. One possible approach is to add a croak to callback.xs to validate that the final byte written to the cursor pointer falls within the code memory allocation, and then confirm that -- without your fix -- any use of the callback trips the error. (I think, at least by inspection, the overwrite is likely to happen regardless of the callback signature, although it's hard for me to say, because some of the callback creation logic is dependent on what the compiler emits for the template function. I was working with a callback that returned void and accepted 6 32-bit words on the stack.) I'm looking at 0.57 source, with a callback.xs source control revision of 109. I apologize because I don't have my development environment set up to build extensions from source, but I think something after line 804 (after the epilogue is written) like the following would help with testing: if (code + tomalloc <= cursor) { croak("CallbackCreate cursor exceeds memory buffer allocation boundary"); } I suppose a WIN32_API_DEBUG printf message might help also, but would presumably not break the test script. Of course, if the code does in fact continue to exhibit an overwrite bug, it might destabilize the extension enough to not run the check (since, by line 804, the corruption is already committed, and could arguably have resulted in a segfault). Anyway, I'm writing this "off the cuff," and I'm by no means an expert in Perl extension development, particularly test scripts. I was actually working on embedding Perl in a Windows application, and was using the Callback extension to enable a script to interface with yet another native Windows library. I've learned a lot about Perl guts, and have climbed some of the learning curve around extension subroutines, but not high enough to be of much help. I'm not sure if this is the sort of guidance for which you were looking, but apart from some defensive programming, I'm not sure how to detect the problem reliably. I was using precompiled binaries from ActiveState, so it's entirely possible the behavior is different with a different runtime heap, a different Perl allocation scheme (ActiveState uses a VMem object attached to CPerlHost to implement PerlMem_malloc), or a different compiler (which might make a different template function, and thus, change how RelocateCode is called). I don't see an obvious way to screen for the problem across compilers and across different implementations of the interpreter, but if you'd like me to give it more thought -- or focus on just ActiveState binaries -- I'm happy to help if I can. Best, Jim James M. Brackett Financial Technology Practice Milliman, Inc. 71 S. Wacker Drive 31st Floor Chicago, IL 60606 Telephone: (312) 726-0677 FAX: (312) 726-5225 E-Mail: jim.brackett@milliman.com "Cosimo Streppone via RT" <bug-Win32-API@rt.cpan.org> 01/13/2009 02:22 PM Please respond to bug-Win32-API@rt.cpan.org To jim.brackett@milliman.com cc Subject [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size <URL: https://rt.cpan.org/Ticket/Display.html?id=42218 > Jim wrote: Show quoted text
> Although I did not recompile from source, I did apply a binary > patch to the DLL to replace the +3 with +4, and the modified binary > worked perfectly, precisely filling the heap block with code, and not > yielding any heap corruption.
Hi Jim, I made the change, but I'd like to add a proper test case for that. Can you send me the example script that caused the problem? Or can you help me build a test case for this bug? Thanks again. ************************************************************************************** This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Unless indicated to the contrary: it does not constitute professional advice or opinions upon which reliance may be made by the addressee or any other party, and it should be considered to be a work in progress. Unless otherwise noted in this email or its attachments, this communication does not form a Statement of Actuarial Opinion under American Academy of Actuaries guidelines. **************************************************************************************
This problem should be fixed in 0.58, release to CPAN already. Jim, can you confirm please?
Subject: Re: [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size
Date: Wed, 4 Feb 2009 09:17:23 -0600
To: bug-Win32-API [...] rt.cpan.org
From: Jim Brackett <jim.brackett [...] milliman.com>
Will do! Thanks for looking into this so promptly. As I said earlier, your work on this module has been exemplary, and I'm sure represents considerable value to the Perl community. James M. Brackett Financial Technology Practice Milliman, Inc. 71 S. Wacker Drive 31st Floor Chicago, IL 60606 Telephone: (312) 726-0677 FAX: (312) 726-5225 E-Mail: jim.brackett@milliman.com From: "Cosimo Streppone via RT" <bug-Win32-API@rt.cpan.org> To: jim.brackett@milliman.com Date: 02/04/2009 03:39 AM Subject: [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size <URL: https://rt.cpan.org/Ticket/Display.html?id=42218 > This problem should be fixed in 0.58, release to CPAN already. Jim, can you confirm please? ************************************************************************************** This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Unless indicated to the contrary: it does not constitute professional advice or opinions upon which reliance may be made by the addressee or any other party, and it should be considered to be a work in progress. Unless otherwise noted in this email or its attachments, this communication does not form a Statement of Actuarial Opinion under American Academy of Actuaries guidelines. **************************************************************************************
Subject: Re: [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size
Date: Thu, 5 Feb 2009 20:15:25 -0600
To: bug-Win32-API [...] rt.cpan.org
From: Jim Brackett <jim.brackett [...] milliman.com>
Cosimo, I've had an opportunity to build version 0.58 of the Win32::API module from CPAN sources under ActiveState, and both a runtime test of my embedded Perl program and a visual inspection of the source code look great. The heap corruption I had previously detected was reconfirmed when running version 0.46 from the ActiveState PPM, but was gone when I dropped in my locally compiled version 0.58. And the addition of 4 bytes for the epilogue in Callback.xs is consistent with what I had implemented as a quick fix already. I think it's safe to say the bug has been vanquished, at least as far as I can tell. I really appreciate your responsiveness and diligence in making this improvement, and again, I offer my thanks for your contributions to the Perl community. If you ever find yourself in need of some help with the module in the future, please at least let me know, as I'd be delighted to lend a hand someday, time permitting. Take care, Jim James M. Brackett Financial Technology Practice Milliman, Inc. 71 S. Wacker Drive 31st Floor Chicago, IL 60606 Telephone: (312) 726-0677 FAX: (312) 726-5225 E-Mail: jim.brackett@milliman.com From: "Cosimo Streppone via RT" <bug-Win32-API@rt.cpan.org> To: jim.brackett@milliman.com Date: 02/04/2009 03:39 AM Subject: [rt.cpan.org #42218] Win32::API::Callback Heap Allocation Size <URL: https://rt.cpan.org/Ticket/Display.html?id=42218 > This problem should be fixed in 0.58, release to CPAN already. Jim, can you confirm please? ************************************************************************************** This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Unless indicated to the contrary: it does not constitute professional advice or opinions upon which reliance may be made by the addressee or any other party, and it should be considered to be a work in progress. Unless otherwise noted in this email or its attachments, this communication does not form a Statement of Actuarial Opinion under American Academy of Actuaries guidelines. **************************************************************************************