Question,How to be compatible with string encoding in c# and rust

[fankaiLiu]

I am new to rust ，c# string encoding is unicode and rust is uft-8. but I don't know how to handle them.
I want to modify string in rust side, and read it in c#

rust code

use std::ffi::{CStr, CString};
use std::fmt::{Display, Formatter, Pointer};
use interoptopus::patterns::result::FFIError;
use interoptopus::patterns::slice::FFISlice;
use interoptopus::{
    ffi_function, ffi_service, ffi_type, function, pattern, Inventory, InventoryBuilder,
};
use interoptopus::{ffi_service_ctor, ffi_service_method};
use libc::c_char;

....

#[ffi_type]
#[repr(C)]
pub struct FLText {
    pub id: u64,
    pub text: *const c_char,
}

#[ffi_function]
pub  extern "C"  fn test_return_str(mut fltext:FLText) -> FLText {
    let c_str = unsafe {
        assert!(!fltext.text.is_null());
        CStr::from_ptr(fltext.text)
    };
    let r_str = c_str.to_str().unwrap().to_string();
    let r_str=r_str.as_bytes();
    // let a_unicode='の'.escape_unicode();
    fltext.text = CString::new(r_str).unwrap().into_raw();
    fltext
}

c# code

                var flText = new FLText();
                string flStr = "你好";
                flText.id = 123123;
                flText.text = GCHandle.Alloc(flStr.ToArray(), GCHandleType.Pinned).AddrOfPinnedObject();
                var result = x.TestReturnStr(flText);
                var resultStr = Marshal.PtrToStringAuto(result.text);

Thanks for advice~

[ralfbiedert]

The very short answer is that "arbitrary" string conversion or editing is out of scope, as the library is primarily concerned with creating bindings.

Although some strings types are supported, e.g., nul-terminated ASCII strings being a C-native concept (and we probably could extend that to UTF-8 slices), modifying them from another language is another caliber.

Two basic issues:

• You'd need special logic to handle the encoding (UTF-16 "Unicode" in this case)
• You can't really handle or communicate that you'd like to change string length as you don't have access to the original (C#) allocator
• And bonus issue for C#, depending on the marshalling attributes C# won't even care about what you do to the string.

I'd recommend to convert the string to UTF-8 in C#, pass that to Rust, and return an entirely different string back to C# (with the implication that each language will have to handle its own allocation).

Even better though might be trying to avoid the situation entirely due to the aforementioned allocation allocation / change-signalling issues.

[fankaiLiu]

Thanks, I'm trying what you said, but not sure it's working. I'm trying to pass in byte[] from c#, but I don't know what to do with the data returned by rust. Should I maintain the length information manually?ORZ

          var flText2 = new FLText();
                flText2.id = 123123;
                string flStr = "你好";
                byte[] encodeBytes = System.Text.Encoding.UTF8.GetBytes(flStr);
                flText2.text = GCHandle.Alloc(encodeBytes, GCHandleType.Pinned).AddrOfPinnedObject();
                var result=x.TestReturnStr(flText2);
                var bytes = StuctArrayFromIntPtr<byte>(result.text, 【No idea of length】);
                string resultString = System.Text.Encoding.UTF8.GetString(bytes);

            public static T[] StuctArrayFromIntPtr<T>(IntPtr outArray, int size) where T : new()
            {
                T[] resArray = new T[size];
                IntPtr current = outArray;
                for (int i = 0; i < size; i++)
                {
                    resArray[i] = new T();
                    Marshal.PtrToStructure(current, resArray[i]);
                    Marshal.DestroyStructure(current, typeof(T));
                    int structsize = Marshal.SizeOf(resArray[i]);
                    current = (IntPtr)((long)current + structsize);
                }
                Marshal.FreeCoTaskMem(outArray);
                return resArray;
            }

Can there be a short examples for me to learn from. :)

[ralfbiedert]

Should I maintain the length information manually?

Yes, you will have to.

Likewise, in your test_return_str() doing fltext.text = CString::new(r_str).unwrap().into_raw(); is not a good idea (unless you're just quickly hacking something together) as you're allocating heap memory and no one knows how to deallocate that.

As a general rule

• do not pass allocation responsibility across FFI boundaries
• This has a corollary, if you allocate on side of the FFI boundary you also need to provide an additional API to free these allocations.
• This in turn has a corollary that your APIs get a bit more awkward / verbose, but that's the price to pay for sane and sound C-level APIs.