Golang Parsing of ProtoBuf - go

I'm new to Golang and I am trying to write an home automation framework in Golang, using the Micro framework and Protobuf framework.
I am currently having a hard time trying to implement a simple registry type service.
An example of the problem I am having is that I have the following, I want to be able to get a list of devices, provided a client does a GET request to http://localhost:8080/view/devices
I have the following protobuf definition:
syntax = "proto3";
service DRegistry {
rpc View(ViewRequest) returns (DeviceRegistry) {}
}
message DeviceRegistry {
repeated Device devices = 1;
}
message ViewRequest {
string Alias = 1;
}
message Device {
string Alias = 1;
string HWAddress = 2;
string WakeUpMethod = 3;
repeated string BoundServices = 4;
}
Then in my service defination I have the following:
package main
import (
"log"
micro "github.com/micro/go-micro"
proto "github.com/srizzling/gotham/proto/device"
"golang.org/x/net/context"
)
// DRegistry stands for Device Registry and is how devices register to Gotham.
type DRegistry struct{}
var devices map[string]proto.Device
func (g *DRegistry) View(ctx context.Context, req *proto.ViewRequest, rsp *proto.DeviceRegistry) error {
filter := req.Alias
devices, err := filterDevices(filter)
rsp.Devices = devices
}
func filterDevices(filter string) (*[]proto.Device, error) {
// Currently only supports listing a single service for now
// TODO: expand filter to be more consise
filteredDevices := make([]proto.Device, 0, len(devices))
for _, e := range devices {
for _, f := range e.BoundServices {
if f == filter {
filteredDevices = append(filteredDevices, e)
}
}
}
return &filteredDevices, nil
}
func main() {
service := micro.NewService(
micro.Name("DRegistry"),
)
proto.RegisterDRegistryHandler(service.Server(), new(DRegistry))
if err := service.Run(); err != nil {
log.Fatal(err)
}
}
The problem I am having is that my IDE (Visual Studio Code) is complianing that I cannot use devices (type *[]device.Device) as type []*device.Device in assignment which is confusing.
TLDR: How do I assign a collection of proto.Devices to the proto.DeviceRegistry?

func filterDevices(filter string) ([]*proto.Device, error) {
// Currently only supports listing a single service for now
// TODO: expand filter to be more consise
filteredDevices := make([]*proto.Device, 0, len(devices))
for _, e := range devices {
for _, f := range e.BoundServices {
if f == filter {
filteredDevices = append(filteredDevices, &e)
}
}
}
return filteredDevices, nil
}
There is a difference between a slice of pointers ([]*) and a pointer to a slice (*[]). You are returning a pointer to slice, whereas what you want is a slice of pointers. We can solve this by:
Updating your filterDevices signature to return a slice of pointers
Updating your make call to make a slice of pointers
Taking the address of e in your call to append (we want a slice of pointers to devices)
Not returning the address of the slice

Related

How does CreateWindowSurface from GLFW work in vulkan-go?

I am trying to create a vk.Surface using go and the github.com/vulkan-go bindings.
Usually the glfw.CreateWindowSurface is declared with four parameters like this CreateWindowSurface(instance interface{}, window *glfw.Window, allocCallbacks unsafe.Pointer, *vk.Surface) so we can declare a variable of type vk.Surface and pass the address of it to the function.
But the glfw.CreateWindowSurface function in vulkan-go is declared like this CreateWindowSurface(instance interface{}, allocCallbacks unsafe.Pointer) and it does not return a vk.Surface but an uintptr of an unsafe.Pointer to a C.VkSurfaceKHR.
I tried drilling thru the pointers and type cast to vk.Surface like this (vk.Surface)(unsafe.Pointer(surface)), but this does not seam to work because when i later pass this to a vk.GetPhysicalDeviceSurfaceSupport function the validation layer outputs the error Validation Error: [ VUID-vkGetPhysicalDeviceSurfaceSupportKHR-surface-parameter ] Object 0: handle = 0x22ce94b8e60, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x801f247e | Invalid VkSurfaceKHR Object 0xc000014148. The Vulkan spec states: surface must be a valid VkSurfaceKHR handle (https://vulkan.lunarg.com/doc/view/1.3.211.0/windows/1.3-extensions/vkspec.html#VUID-vkGetPhysicalDeviceSurfaceSupportKHR-surface-parameter).
This is my code, it's a bit long but maybe it will help you better understand what i am doing:
package main
import (
"bytes"
"errors"
"fmt"
"log"
"unsafe"
"github.com/fijosilo/osies/internal/utils"
"github.com/vulkan-go/glfw/v3.3/glfw"
vk "github.com/vulkan-go/vulkan"
)
const debug bool = true
var width int = 1280
var height int = 720
var vulkanSelectedValidationLayers = []string{}
var vulkanSelectedExtensions = []string{}
var vulkanApplicationInfo *vk.ApplicationInfo = nil
var vulkanCreateInstanceInfo *vk.InstanceCreateInfo = nil
var vulkanDebugReportCallbackCreateInfo *vk.DebugReportCallbackCreateInfo = nil
var vulkanDebugReportCallback *vk.DebugReportCallback = new(vk.DebugReportCallback)
func main() {
logFile := utils.SetupLogOutputFile()
defer logFile.Close()
// init glfw
initializeGLFW()
// create the window
window := createGLFWWindow(width, height, "OSIES")
// init vulkan
initializeVulkan()
// create vulkan instance
instance := createVulkanInstance(window)
// setup validations
if debug {
vk.CreateDebugReportCallback(instance, getVulkanDebugReportCallbackCreateInfo(), nil, vulkanDebugReportCallback)
}
// window surface
surface := createVulkanSurface(instance, window)
// select physical device
physicalDevice, queueFamiliesIndices := selectVulkanPhysicalDeviceAndQueueFamily(instance, &surface)
// create logical device
logicalDevice := createVulkanLogicalDevice(physicalDevice, queueFamiliesIndices)
var graphycsQueue vk.Queue
vk.GetDeviceQueue(logicalDevice, queueFamiliesIndices["graphics"], 0, &graphycsQueue)
var surfaceQueue vk.Queue
vk.GetDeviceQueue(logicalDevice, queueFamiliesIndices["surface"], 0, &surfaceQueue)
// loop
for ; !window.ShouldClose() ; {
glfw.PollEvents()
}
// cleanup
vk.DestroyDevice(logicalDevice, nil)
vk.DestroySurface(instance, surface, nil)
if debug {
vk.DestroyDebugReportCallback(instance, *vulkanDebugReportCallback, nil)
}
vk.DestroyInstance(instance, nil)
window.Destroy()
glfw.Terminate()
}
/* Initializes the GLFW api.
*/
func initializeGLFW() {
err := glfw.Init()
if err != nil{
log.Println(err)
panic(err)
}
}
/* Creates and returns a GLFW window.
*/
func createGLFWWindow(width int, height int, title string) *glfw.Window {
glfw.WindowHint(glfw.ClientAPI, glfw.NoAPI)
glfw.WindowHint(glfw.Resizable, glfw.False)
window, err := glfw.CreateWindow(width, height, title, nil, nil)
if err != nil{
log.Println(err)
panic(err)
}
return window
}
/* Initializes the Vulkan api. It requires the GLFW api to be initialized.
*/
func initializeVulkan() {
vk.SetGetInstanceProcAddr(glfw.GetVulkanGetInstanceProcAddress())
err := vk.Init()
if err != nil{
log.Println(err)
panic(err)
}
}
/* Adds the layer to the slice of selected layers if it is not on the slice yet.
It requires the Vulkan api to be initialized.
*/
func selectVulkanValidationLayer(selectedLayer string) {
vulkanSelectedValidationLayers = utils.AppendStringIfNotFound(vulkanSelectedValidationLayers, vk.ToString([]byte(selectedLayer)))
}
/* Returns a slice of strings where each string is the name of a selected and supported vulkan validation layer.
It requires the Vulkan api to be initialized.
*/
func getVulkanValidationLayers(selectedLayers []string) []string {
// get supported layers' count
var supportedLayerCount uint32
result := vk.EnumerateInstanceLayerProperties(&supportedLayerCount, nil)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan validation layers count with error code %d\n", result)
return []string{}
}
// get supported layers' properties
supportedLayerProperties := make([]vk.LayerProperties, supportedLayerCount)
result = vk.EnumerateInstanceLayerProperties(&supportedLayerCount, supportedLayerProperties)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan validation layers with error code %d\n", result)
return []string{}
}
// get supported layers' property name to slice of strings
supportedLayers := make([]string, supportedLayerCount)
for n, layer := range supportedLayerProperties {
layer.Deref()
supportedLayers[n] = vk.ToString(layer.LayerName[:])
}
// filter the selected layers from the supported layers
layers := []string{}
for _, supportedLayer := range supportedLayers {
// find layer in selected layers
found := false
for _, selectedLayer := range selectedLayers {
if selectedLayer == supportedLayer {
found = true
break
}
}
// add selected supported layer to the slice of layers
if found {
layers = append(layers, supportedLayer)
}
}
// debug
if debug {
// log selected validation layers
log.Print("Vulkan selected validation layers:\n")
for _, layer := range selectedLayers {
log.Printf(" %s\n", layer)
}
// log supported validation layers
log.Print("Vulkan supported validation layers:\n")
for _, layer := range supportedLayers {
log.Printf(" %s\n", layer)
}
// log selected and supported validation layers
log.Print("Vulkan selected and supported validation layers:\n")
for _, layer := range layers {
log.Printf(" %s\n", layer)
}
}
return layers
}
/* Adds the extension to the slice of selected extensions if it is not on the slice yet.
It requires the Vulkan api to be initialized.
*/
func selectVulkanExtension(selectedExtension string) {
vulkanSelectedExtensions = utils.AppendStringIfNotFound(vulkanSelectedExtensions, vk.ToString([]byte(selectedExtension)))
}
/* Gets the GLFW required vulkan extensions and adds them to the slice off selected vulkan extensions.
It requires the Vulkan api to be initialized.
*/
func selectGLFWRequiredExtensions(window *glfw.Window) {
// get GLFW required vulkan extensions
glfwExtensions := window.GetRequiredInstanceExtensions()
if glfwExtensions == nil {
log.Println("Vulkan is not supported in this system")
panic("Vulkan is not supported in this system")
}
// add glfw required extensions to the slice of selected extensions
for _, extension := range glfwExtensions {
selectVulkanExtension(extension)
}
}
/* Returns a slice of strings where each string is the name of a selected and supported vulkan extension.
It requires the Vulkan api to be initialized.
*/
func getVulkanExtensions(selectedExtensions []string) []string {
// get supported extensions' count
var supportedExtensionCount uint32
result := vk.EnumerateInstanceExtensionProperties("", &supportedExtensionCount, nil)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan extensions count with error code %d\n", result)
return []string{}
}
// get supported extensions' properties
supportedExtensionProperties := make([]vk.ExtensionProperties, supportedExtensionCount)
result = vk.EnumerateInstanceExtensionProperties("", &supportedExtensionCount, supportedExtensionProperties)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan extensions with error code %d\n", result)
return []string{}
}
// get supported supportedExtensions' property name to slice of strings
supportedExtensions := make([]string, supportedExtensionCount)
for n, ext := range supportedExtensionProperties {
ext.Deref()
supportedExtensions[n] = vk.ToString(ext.ExtensionName[:])
}
// filter the selected extensions from the supported extensions
extensions := []string{}
for _, supportedExtension := range supportedExtensions {
// find extension in selected extensions
found := false
for _, selectedExtension := range selectedExtensions {
if selectedExtension == supportedExtension {
found = true
break
}
}
// add selected supported extension to the slice of extensions
if found {
extensions = append(extensions, supportedExtension)
}
}
// debug
if debug {
// log selected extensions
log.Print("Vulkan selected extensions:\n")
for _, extension := range selectedExtensions {
log.Printf(" %s\n", extension)
}
// log supported extensions
log.Print("Vulkan supported extensions:\n")
for _, extension := range supportedExtensions {
log.Printf(" %s\n", extension)
}
// log selected and supported extensions
log.Print("Vulkan selected and supported extensions:\n")
for _, extension := range extensions {
log.Printf(" %s\n", extension)
}
}
return extensions
}
/* Returns a pointer to the application info struct.
It requires the Vulkan api to be initialized.
*/
func getVulkanApplicationInfo() *vk.ApplicationInfo {
if vulkanApplicationInfo == nil {
// create the application info struct
vulkanApplicationInfo = new(vk.ApplicationInfo)
vulkanApplicationInfo.SType = vk.StructureTypeApplicationInfo
vulkanApplicationInfo.PNext = nil
vulkanApplicationInfo.PApplicationName = "OSIES"
vulkanApplicationInfo.ApplicationVersion = vk.MakeVersion(1, 0, 0)
vulkanApplicationInfo.PEngineName = "No Engine"
vulkanApplicationInfo.EngineVersion = vk.MakeVersion(1, 0, 0)
vulkanApplicationInfo.ApiVersion = vk.ApiVersion10
}
return vulkanApplicationInfo
}
/* Returns a pointer to the instance create info struct.
It requires the Vulkan api to be initialized.
*/
func getVulkanInstanceCreateInfo(window *glfw.Window) *vk.InstanceCreateInfo {
if vulkanCreateInstanceInfo == nil {
// get choosen and supported validation layers
vkLayers := []string{}
if debug {
selectVulkanValidationLayer("VK_LAYER_KHRONOS_validation")
vkLayers = getVulkanValidationLayers(vulkanSelectedValidationLayers)
selectVulkanExtension(vk.ExtDebugUtilsExtensionName)
selectVulkanExtension(vk.ExtDebugReportExtensionName)
}
// add GLFW required vulkan extensions to the slice of selected vulkan extensions
selectGLFWRequiredExtensions(window)
// get choosen and supported extensions
vkExtensions := getVulkanExtensions(vulkanSelectedExtensions)
// create the instance create info struct
vulkanCreateInstanceInfo = new(vk.InstanceCreateInfo)
vulkanCreateInstanceInfo.SType = vk.StructureTypeInstanceCreateInfo
vulkanCreateInstanceInfo.PNext = nil
vulkanCreateInstanceInfo.Flags = 0
vulkanCreateInstanceInfo.PApplicationInfo = getVulkanApplicationInfo()
vulkanCreateInstanceInfo.EnabledLayerCount = uint32(len(vkLayers))
vulkanCreateInstanceInfo.PpEnabledLayerNames = vkLayers
vulkanCreateInstanceInfo.EnabledExtensionCount = uint32(len(vkExtensions))
vulkanCreateInstanceInfo.PpEnabledExtensionNames = vkExtensions
}
return vulkanCreateInstanceInfo
}
/* Returns a slice of strings where each string is the name of a vulkan extension available on the current system.
It requires the Vulkan api to be initialized.
*/
func createVulkanInstance(window *glfw.Window) vk.Instance {
// create vulkan instance
var instance vk.Instance
result := vk.CreateInstance(getVulkanInstanceCreateInfo(window), nil, &instance)
if result != vk.Success {
log.Printf("Failed to create vulkan instance with error code %d\n", result)
panic("Failed to create vulkan instance")
}
return instance
}
/* Callback function to log vulkan debug messages.
It requires the Vulkan api to be initialized.
*/
var vulkanDebugCallback vk.DebugReportCallbackFunc = func(flags vk.DebugReportFlags, objectType vk.DebugReportObjectType, object uint64, location uint,
messageCode int32, layerPrefix string, message string, userData unsafe.Pointer) vk.Bool32 {
log.Printf("Vulkan: %s\n", message)
return vk.False
}
/* Returns a pointer to the debug report callback create info struct.
It requires the Vulkan api to be initialized.
*/
func getVulkanDebugReportCallbackCreateInfo() *vk.DebugReportCallbackCreateInfo {
if vulkanDebugReportCallbackCreateInfo == nil {
// create the application info struct
vulkanDebugReportCallbackCreateInfo = new(vk.DebugReportCallbackCreateInfo)
vulkanDebugReportCallbackCreateInfo.SType = vk.StructureTypeDebugReportCallbackCreateInfo
vulkanDebugReportCallbackCreateInfo.PNext = nil
vulkanDebugReportCallbackCreateInfo.Flags = vk.DebugReportFlags(vk.DebugReportPerformanceWarningBit | vk.DebugReportWarningBit | vk.DebugReportErrorBit)
vulkanDebugReportCallbackCreateInfo.PfnCallback = vulkanDebugCallback
vulkanDebugReportCallbackCreateInfo.PUserData = nil
}
return vulkanDebugReportCallbackCreateInfo
}
/* Crete and return a window surface.*/
func createVulkanSurface(instance vk.Instance, window *glfw.Window) vk.Surface {
surface, err := window.CreateWindowSurface(instance, nil)
if err != nil {
err := "Vulkan: failed to create a surface"
log.Println(err)
panic(err)
}
// vk.Surface(unsafe.Pointer(surface))
return (vk.Surface)(unsafe.Pointer(surface))
}
/* Finds and returns a slice of physical devices available on the system.*/
func getVulkanPhysicalDevices(instance vk.Instance) []vk.PhysicalDevice {
// get physical devices count
var deviceCount uint32
result := vk.EnumeratePhysicalDevices(instance, &deviceCount, nil)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan physical devices count with error code %d\n", result)
}
// get physical devices
devices := make([]vk.PhysicalDevice, deviceCount)
result = vk.EnumeratePhysicalDevices(instance, &deviceCount, devices)
if result != vk.Success {
log.Printf("VULKAN: Failed to retrieve vulkan physical devices with error code %d\n", result)
}
return devices
}
/* Filters physical devices that have the properties and features that our app needs.*/
func filterVulkanPropertiesAndFeaturesCompatiblePhysicalDevices(devices []vk.PhysicalDevice) []vk.PhysicalDevice {
supportedDevices := []vk.PhysicalDevice{}
// iterate devices
log.Print("Vulkan physical devices available:\n")
for _, device := range devices {
// get device properties
var deviceProperties vk.PhysicalDeviceProperties
vk.GetPhysicalDeviceProperties(device, &deviceProperties)
deviceProperties.Deref()
// log device
deviceName := string(bytes.Split(deviceProperties.DeviceName[:], []byte{0})[0])
log.Printf(" {name: %s, type: %d, apiVersion: %d, driverVersion: %d}\n", deviceName, deviceProperties.DeviceType, deviceProperties.ApiVersion, deviceProperties.DriverVersion)
// get device features
var deviceFeatures vk.PhysicalDeviceFeatures
vk.GetPhysicalDeviceFeatures(device, &deviceFeatures)
deviceFeatures.Deref()
// check if device is compatible
if deviceProperties.DeviceType == vk.PhysicalDeviceTypeDiscreteGpu &&
deviceFeatures.GeometryShader == vk.True {
supportedDevices = append(supportedDevices, device)
}
}
return supportedDevices
}
/* Finds and returns the first physical device that supports the queue families required by our app*/
func getVulkanQueueFamiliesCompatiblePhysicalDevice(devices []vk.PhysicalDevice, surface *vk.Surface) (vk.PhysicalDevice, map[string]uint32, error) {
// iterate devices
for _, device := range devices {
// get queue families' count
var queueFamiliesCount uint32
vk.GetPhysicalDeviceQueueFamilyProperties(device, &queueFamiliesCount, nil)
// get queue families
queueFamilies := make([]vk.QueueFamilyProperties, queueFamiliesCount)
vk.GetPhysicalDeviceQueueFamilyProperties(device, &queueFamiliesCount, queueFamilies)
// check if device supports all the required queue families
selectedQueueFamiliesIndices := map[string]uint32{}
var queueFlags vk.QueueFlags = vk.QueueFlags(vk.QueueGraphicsBit)
var queueCount uint32 = 1
var surfaceSupport vk.Bool32 = vk.False
for i, family := range queueFamilies {
family.Deref()
supportsGraphics := family.QueueFlags & queueFlags == queueFlags && family.QueueCount >= queueCount
supportsSurface := vk.GetPhysicalDeviceSurfaceSupport(device, uint32(i), *surface, &surfaceSupport) == vk.Success
// prefer a single queue family that supports all the requirements
if supportsGraphics && supportsSurface {
selectedQueueFamiliesIndices["graphics"] = uint32(i)
selectedQueueFamiliesIndices["surface"] = uint32(i)
break
}
// otherwise get multiple queue families
_, ok := selectedQueueFamiliesIndices["graphics"]
if !ok && supportsGraphics {
selectedQueueFamiliesIndices["graphics"] = uint32(i)
}
_, ok = selectedQueueFamiliesIndices["surface"]
if !ok && supportsSurface {
selectedQueueFamiliesIndices["surface"] = uint32(i)
}
}
// if the device supports all the required queue families exit early
if len(selectedQueueFamiliesIndices) == 2 {
return device, selectedQueueFamiliesIndices, nil
}
}
return nil, nil, errors.New("Vulkan: failed to find a physical device that supports the queue families required by this app")
}
/* Selects a physical device and queue family compatible with vulkan and the requirements of our app*/
func selectVulkanPhysicalDeviceAndQueueFamily(instance vk.Instance, surface *vk.Surface) (vk.PhysicalDevice, map[string]uint32) {
// get available devices
physicalDevices := getVulkanPhysicalDevices(instance)
if len(physicalDevices) < 1 {
err := "Vulkan: failed to find a device compatible with vulkan"
log.Println(err)
panic(err)
}
// filter devices with the properties and features required by our app
filteredPhysicalDevices := filterVulkanPropertiesAndFeaturesCompatiblePhysicalDevices(physicalDevices)
if len(filteredPhysicalDevices) < 1 {
err := "Vulkan: failed to find a device compatible with the properties and features required by our app"
log.Println(err)
panic(err)
}
// find the first device that supports the queue families required by our app
selectedPhysicalDevice, selectedQueueFamiliesIndices, err := getVulkanQueueFamiliesCompatiblePhysicalDevice(filteredPhysicalDevices, surface)
if err != nil {
err := "Vulkan: failed to find a device compatible that supports the queue families required by our app"
log.Println(err)
panic(err)
}
return selectedPhysicalDevice, selectedQueueFamiliesIndices
}
/* Returns a pointer to the device queue create info struct.*/
func getVulkanDeviceQueueCreateInfo(queueFamilyIndex uint32) *vk.DeviceQueueCreateInfo {
// create the device queue create info struct
deviceQueueCreateInfo := new(vk.DeviceQueueCreateInfo)
deviceQueueCreateInfo.SType = vk.StructureTypeDeviceQueueCreateInfo
deviceQueueCreateInfo.PNext = nil
deviceQueueCreateInfo.Flags = vk.DeviceQueueCreateFlags(0)
deviceQueueCreateInfo.QueueFamilyIndex = queueFamilyIndex
deviceQueueCreateInfo.QueueCount = 1
deviceQueueCreateInfo.PQueuePriorities = []float32{1.0}
return deviceQueueCreateInfo
}
/* Returns a pointer to the device queue create info struct.*/
func getPhysicalDeviceFeatures() *vk.PhysicalDeviceFeatures {
physicalDeviceFeatures := new(vk.PhysicalDeviceFeatures)
return physicalDeviceFeatures
}
/* Returns a pointer to the device queue create info struct.*/
func getVulkanDeviceCreateInfo(deviceQueueCreateInfos []vk.DeviceQueueCreateInfo, physicalDeviceFeatures vk.PhysicalDeviceFeatures) *vk.DeviceCreateInfo {
deviceCreateInfo := new(vk.DeviceCreateInfo)
deviceCreateInfo.SType = vk.StructureTypeDeviceCreateInfo
deviceCreateInfo.PNext = nil
deviceCreateInfo.Flags = vk.DeviceCreateFlags(0)
deviceCreateInfo.QueueCreateInfoCount = uint32(len(deviceQueueCreateInfos))
deviceCreateInfo.PQueueCreateInfos = deviceQueueCreateInfos
// these validation layers are device specific and ignored on modern implementations of vulkan
deviceCreateInfo.EnabledLayerCount = 0
deviceCreateInfo.PpEnabledLayerNames = []string{}
// These extensions are device specific
deviceCreateInfo.EnabledExtensionCount = 0
deviceCreateInfo.PpEnabledExtensionNames = []string{}
deviceCreateInfo.PEnabledFeatures = []vk.PhysicalDeviceFeatures{physicalDeviceFeatures}
return deviceCreateInfo
}
/* Creates and returns a vulkan logical device.*/
func createVulkanLogicalDevice(physicalDevice vk.PhysicalDevice, queueFamiliesIndices map[string]uint32) vk.Device {
// get slice of unique queue families indices
var uniqueQueueFamiliesIndices []uint32
for _, value := range queueFamiliesIndices {
isUnique := true
for _, index := range uniqueQueueFamiliesIndices {
if index == value {
isUnique = false
break
}
}
if isUnique {
uniqueQueueFamiliesIndices = append(uniqueQueueFamiliesIndices, value)
}
}
// get a slice of queue create info struct, one for each unique queue family
var deviceQueueCreateInfos []vk.DeviceQueueCreateInfo
for _, index := range uniqueQueueFamiliesIndices {
deviceQueueCreateInfos = append(deviceQueueCreateInfos, *getVulkanDeviceQueueCreateInfo(index))
}
// get physical device features struct
physicalDeviceFeatures := getPhysicalDeviceFeatures()
// get device create info struct
deviceCreateInfo := getVulkanDeviceCreateInfo(deviceQueueCreateInfos, *physicalDeviceFeatures)
// create the logical device
var logicalDevice vk.Device
result := vk.CreateDevice(physicalDevice, deviceCreateInfo, nil, &logicalDevice)
if result != vk.Success {
err := fmt.Sprintf("Vulkan: failed to create logical device with error code %d\n", result)
log.Println(err)
panic(err)
}
return logicalDevice
}
So how does CreateWindowSurface from GLFW work in vulkan-go?
In vulkan-go the CreateWindowSurface from GLFW returns an uintptr of an unsafe.Pointer to a vk.Surface so to get that to a vk.Surface we need to:
convert the uintptr to an unsafe.Pointer unsafe.Pointer(surface);
then type cast the resulting unsafe.Pointer to a vk.Surface pointer (*vk.Surface)(unsafe.Pointer(surface));
and finally just grab what the pointer of vk.Surface is pointing to *(*vk.Surface)(unsafe.Pointer(surface)).
We can't type cast a pointer to a data type, we either type cast a pointer to a pointer of the data type we want and then grab what this new pointer points at or we first get what this pointer points at and type cast that into the data type we want.

problem with https://github.com/seehuhn/fortuna or maybe i misunderstand Golang

I am trying to use the https://github.com/seehuhn/fortuna for Golang and implement it into an API.
the problem i am having is when i assign *fortuna.Accumulator and assign it to my App struct i cannot use it outside of the function body it was created in.
see example below.
type App struct {
Config config.Config
RNG *fortuna.Accumulator
Sink chan<- time.Time
}
func New(cfg config.Config) *App {
var acc, err = fortuna.NewRNG(cfg.SeedFileName)
if err != nil {
panic("cannot initialise the RNG: " + err.Error())
}
defer acc.Close()
app := App{Config: cfg, RNG: acc, Sink: sink}
/// if i use app.RNG.Uint64() <--- using this here works correctly
return &app
package main
import (
"******/*******/app"
)
func main() {
app := app.New(cfg)
app.RNG.Uint64() <--- this causes Generator not seeded yet error
}
So you defering call
acc.Close()
witch effectivly killing RNG.

Generic function with variable input/output types

Just playing with aws sdk for go. When listing resources of different types I tend to have alot of very similar functions like the two in the example bellow.
Is there a way to rewrite them as one generic function that will return a specific type depending on what is passed on as param?
Something like:
func generic(session, funcToCall, t, input) (interface{}, error) {}
currently I have to do this (functionality is the same just types change):
func getVolumes(s *session.Session) ([]*ec2.Volume, error) {
client := ec2.New(s)
t := []*ec2.Volume{}
input := ec2.DescribeVolumesInput{}
for {
result, err := client.DescribeVolumes(&input)
if err != nil {
return nil, err
}
t = append(t, result.Volumes...)
if result.NextToken != nil {
input.NextToken = result.NextToken
} else {
break
}
}
return t, nil
}
func getVpcs(s *session.Session) ([]*ec2.Vpc, error) {
client := ec2.New(s)
t := []*ec2.Vpc{}
input := ec2.DescribeVpcsInput{}
for {
result, err := client.DescribeVpcs(&input)
if err != nil {
return nil, err
}
t = append(t, result.Vpcs...)
if result.NextToken != nil {
input.NextToken = result.NextToken
} else {
break
}
}
return t, nil
}
Because you only deal with functions it is possible to use the reflect package to generate functions at runtime.
Using the object type (Volume, Vpc) it is possible to derive all subsequents information to provide a fully generic implementation that is really dry, at the extent at the being more complex and slower.
It is untested, you are welcome to help in testing and fixing it, but something like this should put you on the track
https://play.golang.org/p/mGjtYVG2OZS
The registry idea come from this answer https://stackoverflow.com/a/23031445/4466350
for reference the golang documentation of the reflect package is at https://golang.org/pkg/reflect/
package main
import (
"errors"
"fmt"
"reflect"
)
func main() {
fmt.Printf("%T\n", getter(Volume{}))
fmt.Printf("%T\n", getter(Vpc{}))
}
type DescribeVolumesInput struct{}
type DescribeVpcs struct{}
type Volume struct{}
type Vpc struct{}
type Session struct{}
type Client struct{}
func New(s *Session) Client { return Client{} }
var typeRegistry = make(map[string]reflect.Type)
func init() {
some := []interface{}{DescribeVolumesInput{}, DescribeVpcs{}}
for _, v := range some {
typeRegistry[fmt.Sprintf("%T", v)] = reflect.TypeOf(v)
}
}
var errV = errors.New("")
var errType = reflect.ValueOf(&errV).Elem().Type()
var zeroErr = reflect.Zero(reflect.TypeOf((*error)(nil)).Elem())
var nilErr = []reflect.Value{zeroErr}
func getter(of interface{}) interface{} {
outType := reflect.SliceOf(reflect.PtrTo(reflect.TypeOf(of)))
fnType := reflect.FuncOf([]reflect.Type{reflect.TypeOf(new(Session))}, []reflect.Type{outType, errType}, false)
fnBody := func(input []reflect.Value) []reflect.Value {
client := reflect.ValueOf(New).Call(input)[0]
t := reflect.MakeSlice(outType, 0, 0)
name := fmt.Sprintf("Describe%TsInput", of)
descInput := reflect.New(typeRegistry[name]).Elem()
mName := fmt.Sprintf("Describe%Ts", of)
meth := client.MethodByName(mName)
if !meth.IsValid() {
return []reflect.Value{
t,
reflect.ValueOf(fmt.Errorf("no such method %q", mName)),
}
}
for {
out := meth.Call([]reflect.Value{descInput.Addr()})
if len(out) > 0 {
errOut := out[len(out)-1]
if errOut.Type().Implements(errType) && errOut.IsNil() == false {
return []reflect.Value{t, errOut}
}
}
result := out[1]
fName := fmt.Sprintf("%Ts", of)
if x := result.FieldByName(fName); x.IsValid() {
t = reflect.AppendSlice(t, x)
} else {
return []reflect.Value{
t,
reflect.ValueOf(fmt.Errorf("field not found %q", fName)),
}
}
if x := result.FieldByName("NextToken"); x.IsValid() {
descInput.FieldByName("NextToken").Set(x)
} else {
break
}
}
return []reflect.Value{t, zeroErr}
}
fn := reflect.MakeFunc(fnType, fnBody)
return fn.Interface()
}
Proxying 3rd party API, is quite simple to implement with
go, here is how' it got implemented with endly e2e test runner AWS proxy
I would say that AWS API is perfect candidate for proxying, as long as reflection performance price is not an issue.
Some other 3rd party API like kubernetes
are much more challenging, but still quite easy to proxy with go, which is a combination of reflection and code generation:

Adding custom layer to packet from capture fails

I am trying to implement my own decoding layer ontop of TCP, so far it only works when I create a packet without any Eth/IP/TCP header and set its layer to my custom layer manually. The data of the custom protocol is inside an ordinary TCP payload.
How do I decode only the payload of the TCP layer as another layer?
package main
import (
"fmt"
"github.com/google/gopacket"
"github.com/google/gopacket/pcap"
)
var (
pcapFile string = "capt.pcap"
handle *pcap.Handle
err error
)
type CustomLayer struct {
SomeByte byte
AnotherByte byte
restOfData []byte
}
var CustomLayerType = gopacket.RegisterLayerType(
2001,
gopacket.LayerTypeMetadata{
"CustomLayerType",
gopacket.DecodeFunc(decodeCustomLayer),
},
)
func (l CustomLayer) LayerType() gopacket.LayerType {
return CustomLayerType
}
func (l CustomLayer) LayerContents() []byte {
return []byte{l.SomeByte, l.AnotherByte}
}
func (l CustomLayer) LayerPayload() []byte {
return l.restOfData
}
func decodeCustomLayer(data []byte, p gopacket.PacketBuilder) error {
p.AddLayer(&CustomLayer{data[0], data[1], data[2:]})
// nil means this is the last layer. No more decoding
return nil
}
func main() {
handle, err = pcap.OpenOffline(pcapFile)
if err != nil {
log.Fatal(err)
}
defer handle.Close()
packetSource := gopacket.NewPacketSource(handle, handle.LinkType())
for packet := range packetSource.Packets() {
tcpLayer := packet.Layer(layers.LayerTypeTCP)
if tcpLayer != nil {
fmt.Println("TCP layer detected.")
tcp, _ := tcpLayer.(*layers.TCP)
fmt.Println("Sequence number: ", tcp.Seq)
customLayer := packet.Layer(CustomLayerType)
if customLayer != nil { // always nil
customLayerContent, _ := customLayer.(*CustomLayer)
// Now we can access the elements of the custom struct
fmt.Println("Payload: ", customLayerContent.LayerPayload())
fmt.Println("SomeByte element:", customLayerContent.SomeByte)
fmt.Println("AnotherByte element:", customLayerContent.AnotherByte)
}
}
fmt.Println()
}
}
Most of the code is from this great post by devdungeon.
As no one responded I am going to answer it myself now.
Basically we have 3 options to handle this:
Create an extended TCP layer that handles our additional bytes and override default one by setting layers.LinkTypeMetadata[layers.LinkTypeTCP] to our extended version. Have a look at this example.
Create a new packet from the TCP payload using gopacket.NewPacket setting firstLayerDecoder to CustomLayerType and decode it normally.
As you mostly don't need an actual layer but instead a filled CustomLayer struct simply write a DecodeBytesToCustomStruct function where you pass TCP payload. This way we can even return multiple structs from one packets payload which wouldn't be possible otherwise.
Omit all CustomLayer code from above.
type CustomStruct struct {
SomeByte byte
AnotherByte byte
restOfData []byte
}
func (customStruct *CustomStruct) DecodeStructFromBytes(data []byte) error {
customStruct.SomeByte = data[0]
customStruct.AnotherByte = data[1]
customStruct.restOfData = data[2:]
return nil
}
In your main.go
for packet := range packetSource.Packets() {
tcpLayer := packet.Layer(layers.LayerTypeTCP)
if tcpLayer != nil {
tcp, _ := tcpLayer.(*layers.TCP)
if tcp.Payload != nil && len(tcpLayer.Payload) > 0 {
customStruct := CustomStruct{}
customStruct.DecodeStructFromBytes(tcp.Payload)
fmt.Println("SomeByte element:", customStruct.SomeByte)
}
}
}
tcp.Payload is the same as packet.ApplicationLayer().Payload()

Golang RPC encode custom function

I am trying to use github.com/dullgiulio/pingo and send my custom struct
type LuaPlugin struct {
Name string
List []PluginTable
}
type PluginTable struct {
Name string
F lua.LGFunction
}
// LoadPlugins walks over the plugin directory loading all exported plugins
func LoadPlugins() {
//
p := pingo.NewPlugin("tcp", "plugins/test")
// Actually start the plugin
p.Start()
// Remember to stop the plugin when done using it
defer p.Stop()
gob.Register(&LuaPlugin{})
gob.Register(&PluginTable{})
var resp *LuaPlugin
// Call a function from the object we created previously
if err := p.Call("MyPlugin.SayHello", "Go developer", &resp); err != nil {
log.Print(err)
} else {
log.Print(resp.List[0])
}
}
However I am always getting nil for the F field of ym struct. This is what I am sending on the client
// Create an object to be exported
type MyPlugin struct{}
// Exported method, with a RPC signature
func (p *MyPlugin) SayHello(name string, msg *util.LuaPlugin) error {
//
//
*msg = util.LuaPlugin{
Name: "test",
List: []util.PluginTable{
{
Name: "hey",
F: func(L *lua.LState) int {
log.Println(L.ToString(2))
return 0
},
},
},
}
return nil
}
Is it not possible to send custom data types over RPC?
I'm not familiar with the library however, you could try converting the struct to a byte slice before transport. Late reply, might help others....
Simple conversion: returns a struct as bytes
func StructToBytes(s interface{}) (converted []byte, err error) {
var buff bytes.Buffer
encoder := gob.NewEncoder(&buff)
if err = encoder.Encode(s); err != nil {
return
}
converted = buff.Bytes()
return
}
Decoder: returns a wrapper to decode the bytes into
func Decoder(rawBytes []byte) (decoder *gob.Decoder) {
reader := bytes.NewReader(rawBytes)
decoder = gob.NewDecoder(reader)
return
}
Example:
type MyStruct struct {
Name string
}
toEncode := MyStruct{"John Doe"}
// convert the struct to bytes
structBytes, err := StructToBytes(toEncode)
if err != nil {
panic(err)
}
//-----------
// send over RPC and decode on other side
//-----------
// create a new struct to decode into
var DecodeInto MyStruct
// pass the bytes to the decoder
decoder := Decoder(structBytes)
// Decode into the struct
decoder.Decode(&DecodeInto)
fmt.Println(DecodeInto.Name) // John Doe
Due to the use of the gob package you can swap types and typecast to a certain extent.
For more see: https://golang.org/pkg/encoding/gob/

Resources