Tom’s Blog

May 11, 2009

A P/Invoke Primer for the Experienced VB6 Programmer

Filed under: .NET,P/Invoke,Visual Basic — Tom Shelton @ 10:52 pm

"In VB6 I used to use SomeSuchAPI to perform task X – I have tried to use SomeSuchAPI in my new SuperVelocityStrumCruncher app written in VB.NET, but it won’t work!  Help!"  And on goes the refrain. 

When I see these types of questions, nine times out of ten, I know the answer before I even read the question.  Or, that is to say, I can pretty much surmise that the answer will fall into one of a couple of typical traps that the experienced VB6 user faces when deciding to use P/Invoke in VB.NET (which, is a pretty relevant question because many times the task you were using the API for in VB6, may just be taken care of by the .NET framework).

So, I’m going to cover a few tips that should help the experienced VB6 user successfully make use of Windows or other API calls from VB.NET.

Know Your Data Types

That’s right – know your data types.  For better or for worse, many of the integral data types that you used to know and love in VB6 have changed sizes in VB.NET.  For example, Long is no longer a 32-bit Integer – it’s 64-bit.  So, forget what you knew about integer types and their sizes in VB6 and learn them a new in VB.NET.

Here is a quick conversion table that might help speed along this task:

Size/Sign VB6 Data Type VB.NET Data Type
8 bits/unsigned Byte Byte
8 bits/signed N/A SByte
16 bits/unsigned N/A UShort
16 bits/signed Integer Short
32 bits/unsigned N/A UInteger
32 bits/signed Long Integer
64 bits/unsigned N/A ULong
64 bits/signed N/A Long
This table is based on the information found here.

So…  What does all of this mean in the actual context of a Windows API call?  Well, lets look at the GetWindowsDirectory API call.  A typical declaration for this in VB6 might be:

   1: Private Declare Function GetWindowsDirectory Lib "KERNEL32" Alias "GetWindowsDirectoryA" _
   2:     (ByVal lpBuffer As String, ByVal nSize As Long) As Long

This call is incorrect in VB.NET – we can see that we are passing a Long value to the nSize parameter, which is a 32-bit integer in VB6, but is 64-bit in VB.NET.  This can cause problems as it can unbalance the functions call stack.  To correct this we can declare the function in VB.NET like so:

   1: Private Declare Function GetWindowsDirectory Lib "KERNEL32" Alias "GetWindowsDirectoryA" _
   2:     (ByVal lpBuffer As String, ByVal nSize As Integer) As Integer

If you were to use this declaration in VB.NET, it would work – but, there are a couple of other levels of enlightenment that can be achieved here on our way to API Nirvana! 

Drop The Alias

Any one else remember the bad old days when a seemingly simple call to an API call from VB could cause gyrations such as these to occur on NT based OS’s?


Good Times!  Good Times!

Well, fortunately, the VB.NET marshaller has improved significantly when it comes to handling of strings.  For instance, it can call the W functions directly without the need for hacks using StrPtr or creating custom API typelibs.  And, as if that wasn’t good news, it actually gets better!  You don’t even have to alias your functions anymore. 

The venerable old Declare statement has a new option in VB.NET that lets you declare the character set that should be used when calling the function.  There are 3 choices – Ansi, Unicode, and Auto.  When dealing with the A/W function set of the Windows API, Auto is often a good choice because you can just declare the function and let the .NET runtime worry about if it should call the A or the W version.

Here’s our function declaration again with a bit more .NET goodness:

   1: Private Declare Auto Function GetWindowsDirectory Lib "KERNEL32" _
   2:     (ByVal lpBuffer As String, ByVal nSize As Integer) As Integer

Isn’t that nice!

Mind Your Buffers

Ok, I want to point out one more minor adjustment that probably should be made to our Declare statement.  In the name of correctness, since the buffer we are passing in to this function is intended to be filled by the API call, we should not be passing the lpBuffer parameter as String.  See in a nod to compatibility with it’s predecessor, the VB.NET marshaller – unlike C#’s – does not respect the immutability of strings and will go through the extra gyrations necessary to make sure you get the result you have come to expect in your long association with VB.CLASSIC.  With out getting into arguments about computability vs correctness – I’m simply going to show you the recommended way of declaring string buffers that are intended to be filled by called API’s and you are free to make your own decision as to which way to go with this:

   1: Private Declare Auto Function GetWindowsDirectory Lib "KERNEL32" _
   2:     (ByVal lpBuffer As System.Text.StringBuilder, ByVal nSize As Integer) As Integer

Keep in mind, the use of StringBuilder is only recommended when the string is intended to be modified by the called function – if it’s a simple constant string buffer, then String is definitely the appropriate type.  Here is a complete sample using our declare statement:

   1: Option Strict On
   2: Option Explicit On
   4: Imports System
   5: Imports System.Text
   7: Module Module1
   8:     Private Const MAX_PATH As Integer = 260
   9:     Private Declare Auto Function GetWindowsDirectory Lib "KERNEL32" _
  10:         (ByVal lpBuffer As StringBuilder, ByVal nSize As Integer) As Integer
  12:     Sub Main()
  13:         Console.WriteLine(GetWindowsDirectory())
  14:     End Sub
  16:     Private Function GetWindowsDirectory() As String
  17:         Dim buffer As New StringBuilder(MAX_PATH)
  18:         GetWindowsDirectory(buffer, buffer.Capacity)
  19:         Return buffer.ToString()
  20:     End Function
  21: End Module

Handle Your Handles

The last bit of advice that I’m going to render in this article is about handles.  In the Windows API the concept of handles is very important, and handles are used all over the place.  When ever you are dealing with a handle type (such as HWND), then it is recommended that you use System.IntPtr as the type for that parameter over Integer.  The main reason for this is that the IntPtr is a wrapper over an unsigned system sized integer – so that means that it is 32-bit on 32-bit systems and 64-bit on 64-bit systems.  See how that works?  Anyway, it might make your porting life in the future easier if you remember that little tip.

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress