Converting byte array to string with correct encoding

Question

I have this bit of C# code that I have translated to VB using http://www.developerfusion.com/tools/convert/csharp-to-vb/

private string DecodeToken (string token, string key)
{           
    byte [] buffer = new byte[0];
    string decoded = "";
    int i;
        if (Scramble (Convert.FromBase64String(token), key, ref buffer))
    {
        for (i=0;i<buffer.Length;i++)
        {
            decoded += Convert.ToString((char)buffer[i]);
        }
    }
    return(decoded);
}

Which, after a little modification, gives this:

Private Function DecodeToken(token As String, key As String) As String
    Dim buffer As Byte()
    Dim decoded As String = ""
    Dim index As Integer
    If Scramble(Convert.FromBase64String(token), key, buffer) Then
        For index = 0 To buffer.Length - 1
            decoded += Convert.ToString(ChrW(buffer(index)))
        Next
        'decoded = UTF8Encoding.ASCII.GetString(pbyBuffer)
        'decoded = UnicodeEncoding.ASCII.GetString(pbyBuffer)
        'decoded = ASCIIEncoding.ASCII.GetString(pbyBuffer)
    End If
    Return decoded
End Function

Scramble just rearranges the array in a specific way and I've checked the VB and C# outputs against each other so it can be ignored. It's inputs and outputs are byte arrays so it shouldn't affect the encoding.

The problem lies in that the result of this function is fed into a hashing algorithm which is then compared against the hashing signature. The result of the VB version, when hashed, does not match to the signature.

You can see from the comments that I've attempted to use different encodings to get the byte buffer out as a string but none of these have worked.

The problem appears to lie in the transalation of decoded += Convert.ToString((char)buffer[i]); to decoded += Convert.ToString(ChrW(buffer(index))).

Does ChrW produce the same result as casting as a char and which encoding will correctly duplicate the reading of the byte array?

Edit: I always have Option Strict On but it's possible that the original C# doesn't so it may be affected by implicit conversion. What does the compiler do in that situation?

I know this isn't a code review, but use a StringBuilder! It might even fix your problem. — Tim Rogers
– Tim Rogers, Commented Oct 2, 2012 at 16:09
As an obvious question you say the hash is different - I presume you have inspected the two strings side by side to see if there are obvious differences? This should pretty much point you to what the problem is I'd have thought (though you may need to inspect in detail since a line break being CR instead of CRLF may not be visible at a glance). — Chris
– Chris, Commented Oct 2, 2012 at 16:22
@TimRogers - The C# is third party code so I just transalted it as directly as possible, haven't refactored yet although thats because I was expecting to use an Encoding.xxx.GetString method — Carl Onager
– Carl Onager, Commented Oct 3, 2012 at 6:41
If you show us the Encode method, I can suggest an optimal decode implementation. Failing that samples of keys may suffice. — Jodrell
– Jodrell, Commented Oct 3, 2012 at 9:36
@Jodrell - The encode code is VB6 translated to VB.NET and then rewritten. To give you a proper sample would take up an unreasonable amount of space for a question. I'll see what I can do though — Carl Onager
– Carl Onager, Commented Oct 3, 2012 at 10:16

Jodrell · Accepted Answer · 2012-10-03 09:48:51Z

3

Quick answer

decoded += Convert.ToString((char)buffer[i]);

is equivalent to

decoded &= Convert.ToString(Chr(buffer[i]));

VB.Net stops you taking the hacky approach used in the c# code, a Char is Unicode so consists of two bytes.

This looks likes a better implementation of what you have.

Private Function DecodeToken(encodedToken As String, key As String) As String
    Dim scrambled = Convert.FromBase64String(encodedToken)
    Dim buffer As Byte()
    Dim index As Integer

    If Not Scramble(scrambled, key, buffer) Then
        Return Nothing
    End If

    Dim descrambled = new StringBuilder(buffer.Length);

    For index = 0 To buffer.Length - 1
        descrambled.Append(Chr(buffer(index)))
    Next

    Return descrambled.ToString()
End Function

edited Oct 3, 2012 at 9:48

answered Oct 3, 2012 at 9:30

Jodrell

35.8k6 gold badges94 silver badges131 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Chris · Accepted Answer · 2012-10-02 16:10:55Z

0

have you tried the most direct code translation:

decoded += Convert.ToString(CType(buffer[i], char))

When covnerting a byte array to a string you should really make sure you know the encoding first though. If this is set in whatever is providing the byte array then you should use that to decode the string.

For more details on the ChrW (and Chr) functions look at http://msdn.microsoft.com/en-us/library/613dxh46%28v=vs.80%29.aspx . In essence ChrW assumes that the passed int is a unicode codepoint which may not be a valid assumption (I believe from 0 to 127 this wouldn't matter but the upper half of the byte might be different). if this is the problem then it will likely be accented and other such "special" characters that are causing the problem.

answered Oct 2, 2012 at 16:10

Chris

27.7k7 gold badges77 silver badges94 bronze badges

1 Comment

Carl Onager Over a year ago

Compiler error: 'Byte' values cannot be converted to 'Char'. Use 'Microsoft.VisualBasic.ChrW' to interpret a numeric value as a Unicode character or first convert it to 'String' to produce a digit.

Chris · Accepted Answer · 2012-10-03 09:04:41Z

0

Give the following a go:

decoded += Convert.ToChar(foo)

It will work (unlike my last attempt that made assumptions about implicit conversions being framework specific and not language specific) but I can't guarantee that it will be the same as the .NET.

Given you say in comments you expected to use Encoding.xxx.GetString then why don't you use that? Do you know what the encoding was in the original string to byte array? If so then just use that. It is the correct way to convert a byte array to a string anyway since doing it byte by byte will definitely break for any multi-byte characters (clearly).

edited Oct 3, 2012 at 9:04

Chris

27.7k7 gold badges77 silver badges94 bronze badges

answered Oct 2, 2012 at 16:09

J. Tanner

5752 silver badges10 bronze badges

1 Comment

Carl Onager Over a year ago

Compiler error: 'Byte' values cannot be converted to 'Char'. Use 'Microsoft.VisualBasic.ChrW' to interpret a numeric value as a Unicode character or first convert it to 'String' to produce a digit.

Ralph Esslinger · Accepted Answer · 2014-02-06 00:06:17Z

0

A small improvement

Private Function DecodeToken(encodedToken As String, key As String) As String
    Dim scrambled = Convert.FromBase64String(encodedToken)
    Dim buffer As Byte()
    Dim index As Integer

    If Not Scramble(scrambled, key, buffer) Then
        Return Nothing
    End If

    Dim descrambled = System.Text.Encoding.Unicode.GetString(buffer, 0, buffer.Length);

    Return descrambled
End Function

answered Feb 6, 2014 at 0:06

Ralph Esslinger

415 bronze badges

Collectives™ on Stack Overflow

Converting byte array to string with correct encoding

4 Answers 4

Comments

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related