Conversions between Codon and C/C++ types

[NK-MXD]

I'm trying to use Codon C++ integration, and I can call codon function in cpp successfully. However, I'm confused how to get the codon function return value in C++ code.

There is a example. I want to get function foo and get_matrix returned array value in C++, but I can't find the conversion from codon array to any C++ type.

from python import numpy as np
from python import torch

@export
def foo(n: int):
    x = np.array([1, 2, 3, 4]) * 10
    print(x)
    return x

@export
def get_matrix(n: int, m: int):
    x = torch.rand(n, m)
    print(x)
    return x

#include <stdint.h>
#include <stdio.h>

extern "C" int64_t* foo(int64_t);
extern "C" int64_t** get_matrix(int64_t, int64_t);

int main() {
  int64_t* foo_res = foo(4);
  for(int i = 0; i < 4; i++) {
    printf("%ld\n", foo_res[i]);
  }

  int64_t** get_matrix_res = get_matrix(2, 3);
  return 0;
}

Output is wrong.

[10 20 30 40]
140158036388912
0
140158036336240
0
tensor([[0.4818, 0.3781, 0.0784],
        [0.2020, 0.3399, 0.3799]])

Hope for answers as soon as possible. 🙏

[Shriharsh-Deshmukh]

The issue lies in converting Python (Codon) arrays to C++ types. Codon uses numpy arrays and tensors (from PyTorch), which need to be converted to C++ data types. You can't directly assign the Python numpy array or tensor to C++ types.

Here’s a techy workaround:

1. Modify the Codon function to convert the numpy array/tensor to a simple Python list before returning it.

Example:

@export
def foo(n: int):
    x = np.array([1, 2, 3, 4]) * 10
    return x.tolist()

@export
def get_matrix(n: int, m: int):
    x = torch.rand(n, m)
    return x.numpy().tolist()

2. In your C++ code, interpret the return values as arrays of integers or floats, depending on the type of data you expect.

You’ll need to adjust your extern "C" function signatures to reflect that you’re now dealing with simple arrays. Ensure that the data is transferred properly between Python and C++ contexts, avoiding issues with pointers or references.

If you're working with tensors, ensure you manage the memory properly and handle the multi-dimensional arrays correctly on the C++ side.

[NK-MXD]

Thank you for your reply!

If I convert the Python (Codon) arrays to simple list, how can I know the length of the list and how can I take the elements out of it in C++.🙏

[Shriharsh-Deshmukh]

To know the length of the list and access its elements in C++, you'll need to adjust both your Codon and C++ code. Here’s how:

1. Return List and Length from Codon

Modify your Codon function to return both the list and its length.

Example for foo function:

@export
def foo(n: int):
    x = np.array([1, 2, 3, 4]) * 10
    return x.tolist(), len(x)  # Return list and its length

2. Update C++ Function Signature

Change your C++ code to handle both the list and the length.

extern "C" int64_t* foo(int64_t, int64_t*);  // Add a parameter to get the list length

int main() {
    int64_t foo_length;
    int64_t* foo_res = foo(4, &foo_length);  // Get the list and its length

    // Use the length to access elements
    for (int i = 0; i < foo_length; i++) {
        printf("%ld\n", foo_res[i]);  // Print each element
    }

    return 0;
}

How it Works:

• Length: The length of the list is returned alongside the list itself in Codon.
• Accessing Elements: In C++, you use the length to loop through and access the list elements.

This approach ensures you know the length of the list and can access each element correctly in C++.