Writing a Terraform provider with nested map - go

I am writing a custom provider with 2 level deeply nested map. I am able to expand the schema when calling create function. However, I am having issues when I try to set this value from read function after this resource has been created. I tried to follow Terraform documentation steps, "Complex read" section but I am getting error Invalid address to set: []string{"docker_info", "0", "port_mapping"}.
Schema looks like this:
"docker_info": {
Type: schema.TypeList,
Required: true,
MaxItems: 1,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"image": &schema.Schema{
Type: schema.TypeString,
Required: true,
},
"force_pull_image": &schema.Schema{
Type: schema.TypeBool,
Optional: true,
Default: "false",
},
"network": &schema.Schema{
Type: schema.TypeString,
Optional: true,
Default: "BRIDGE",
ValidateFunc: validateDockerNetwork,
},
// We use typeSet because this parameter can be unordered list and must be unique.
"port_mapping": &schema.Schema{
Type: schema.TypeSet,
Optional: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"host_port": &schema.Schema{
Type: schema.TypeInt,
Required: true,
},
"container_port": &schema.Schema{
Type: schema.TypeInt,
Required: true,
},
"container_port_type": &schema.Schema{
Type: schema.TypeString,
Optional: true,
ValidateFunc: validateSingularityPortMappingType,
},
"host_port_type": &schema.Schema{
Type: schema.TypeString,
Optional: true,
ValidateFunc: validateSingularityPortMappingType,
},
"protocol": &schema.Schema{
Type: schema.TypeString,
Optional: true,
ValidateFunc: validateSingularityPortProtocol,
Default: "tcp",
},
},
},
},
},
},
},
in my read function, I have:
d.Set("docker_info", flattenDockerInfo(ContainerInfo.DockerInfo))
func flattenDockerInfo(in singularity.DockerInfo) []interface{} {
var out = make([]interface{}, 0, 0)
m := make(map[string]interface{})
m["network"] = in.Network
m["image"] = in.Image
m["force_pull_image"] = in.ForcePullImage
if len(in.PortMappings) > 0 {
m["port_mapping"] = flattenDockerPortMappings(in.PortMappings)
}
out = append(out, m)
return out
}
func flattenDockerPortMappings(in []singularity.DockerPortMapping []map[string]interface{} {
var out = make([]map[string]interface{}, len(in), len(in))
for i := range in {
m := make(map[string]interface{})
m["container_port"] = v.ContainerPort
m["container_port_type"] = v.ContainerPortType
m["host_port"] = v.HostPort
m["host_port_type"] = v.HostPortType
m["protocol"] = v.Protocol
out[i] = m
}
return out
}
singularityDocker struct:
type DockerInfo struct {
Parameters map[string]string
`json:"parameters,omitempty"`
ForcePullImage bool
`json:"forcePullImage,omitempty"`
SingularityDockerParameters []SingularityDockerParameter
`json:"dockerParameters,omitEmpty"`
Privileged bool
`json:"privileged,omitEmpty"`
Network string
`json:"network,omitEmpty"` //Value can be BRIDGE, HOST, or NONE
Image string
`json:"image"`
PortMappings []DockerPortMapping
`json:"portMappings,omitempty"`
}
type DockerPortMapping struct {
ContainerPort int64 `json:"containerPort"`
ContainerPortType string `json:"containerPortType,omitempty"`
HostPort int64 `json:"hostPort"`
HostPortType string `json:"hostPortType,omitempty"`
Protocol string `json:"protocol,omitempty"`
}
I expect to see something like
"docker_info.0.port_mapping.3218452487.container_port": "8888",
"docker_info.0.port_mapping.3218452487.container_port_type": "LITERAL",
"docker_info.0.port_mapping.3218452487.host_port": "0",
"docker_info.0.port_mapping.3218452487.host_port_type": "FROM_OFFER",
"docker_info.0.port_mapping.3218452487.protocol": "tcp",
I found out that by adding below code to create a typeSet
m["port_mapping"] = schema.NewSet(portMappingHash, []interface{}{flattenDockerPortMappings(in.PortMappings)})
func portMappingHash(v interface{}) int {
var buf bytes.Buffer
x := v.([]map[string]interface{})
for i := range x {
buf.WriteString(fmt.Sprintf("%s-%d", "test", i))
}
return hashcode.String(buf.String())
}
I now get docker_info.0.port_mapping.2384314926: '' expected a map, got 'slice'

Related

How to write a custom terraform provider schema for a map of maps values? [duplicate]

I have a golang function that returns roles of type map[string]map[string]string
eg:
map[foo:map[name:abc env:dev id:465 project:e-1] boo:map[name:def env:prd id:82 project:e-1] :doo[name:ght env:stg id:353 project:e-3]]
and I created a schema for it like the following...
func dataSourceAwsAccountHelper() *schema.Resource {
return &schema.Resource{
Read: accountHelperRead,
Schema: map[string]*schema.Schema{
"roles": {
Type: schema.TypeMap,
Elem: &schema.Schema{
Type: schema.TypeMap,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
Computed: true,
},
"id": &schema.Schema{
Computed: true,
Type: schema.TypeString,
},
},
}
}
And the create method to pass the role values to the schema
func rolesRead(d *schema.ResourceData, m interface{}) error {
filteredRoles := filterAccounts("john") // returns `map[string]map[string]string`
if err := d.Set("account_map", filteredRoles); err != nil {
return err
}
//accountMaps := make(map[string]interface{})
d.SetId("-")
return nil
}
but the Terraform output is an empty map, how do I fix it please help :)
Outputs:
output = {
"roles" = tomap(null) /* of map of string */
"id" = tostring(null)
}
expecting output like
Outputs:
output = {
"roles" = { foo = {name = "abc" env = "dev" id= 465 project = "e-1"}
boo = {name = "efg" env = "prd" id= 82 project = "e-2"}
},
"id" = "-"
}
What you are trying to do here is not possbile with the legacy Terraform SDK that you are using. Maps can only be of primitive types: TypeString, TypeInt, TypeBool.
To create this structure you'll need to migrate to the new framework, which is built for the type system of modern Terraform rather than (as is the case for SDKv2) the type system of classic Terraform v0.11 and earlier.
In the Terraform Plugin Framework, the equivalent structure to what you tried to describe here is MapNestedAttribute, with the following describing the schema structure you showed in your question:
schema.MapNestedAttribute{
NestedObject: schema.NestedAttributeObject{
Attributes: map[string]schema.Attribute{
"name": schema.StringAttribute{
// ...
},
"env": schema.StringAttribute{
// ...
},
"id": schema.NumberAttribute{
// ...
},
"project": schema.StringAttribute{
// ...
},
},
},
}
This represents a map of objects with the given attributes, and so the above schema type is equivalent to the following type constraint as might be written in a Terraform module using the Terraform language's type constraint syntax:
map(
object({
name = string
env = string
id = number
project = string
})
)

How to create a custom terraform datasource provider schema for a map of maps values?

I have a golang function that returns roles of type map[string]map[string]string
eg:
map[foo:map[name:abc env:dev id:465 project:e-1] boo:map[name:def env:prd id:82 project:e-1] :doo[name:ght env:stg id:353 project:e-3]]
and I created a schema for it like the following...
func dataSourceAwsAccountHelper() *schema.Resource {
return &schema.Resource{
Read: accountHelperRead,
Schema: map[string]*schema.Schema{
"roles": {
Type: schema.TypeMap,
Elem: &schema.Schema{
Type: schema.TypeMap,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
Computed: true,
},
"id": &schema.Schema{
Computed: true,
Type: schema.TypeString,
},
},
}
}
And the create method to pass the role values to the schema
func rolesRead(d *schema.ResourceData, m interface{}) error {
filteredRoles := filterAccounts("john") // returns `map[string]map[string]string`
if err := d.Set("account_map", filteredRoles); err != nil {
return err
}
//accountMaps := make(map[string]interface{})
d.SetId("-")
return nil
}
but the Terraform output is an empty map, how do I fix it please help :)
Outputs:
output = {
"roles" = tomap(null) /* of map of string */
"id" = tostring(null)
}
expecting output like
Outputs:
output = {
"roles" = { foo = {name = "abc" env = "dev" id= 465 project = "e-1"}
boo = {name = "efg" env = "prd" id= 82 project = "e-2"}
},
"id" = "-"
}
What you are trying to do here is not possbile with the legacy Terraform SDK that you are using. Maps can only be of primitive types: TypeString, TypeInt, TypeBool.
To create this structure you'll need to migrate to the new framework, which is built for the type system of modern Terraform rather than (as is the case for SDKv2) the type system of classic Terraform v0.11 and earlier.
In the Terraform Plugin Framework, the equivalent structure to what you tried to describe here is MapNestedAttribute, with the following describing the schema structure you showed in your question:
schema.MapNestedAttribute{
NestedObject: schema.NestedAttributeObject{
Attributes: map[string]schema.Attribute{
"name": schema.StringAttribute{
// ...
},
"env": schema.StringAttribute{
// ...
},
"id": schema.NumberAttribute{
// ...
},
"project": schema.StringAttribute{
// ...
},
},
},
}
This represents a map of objects with the given attributes, and so the above schema type is equivalent to the following type constraint as might be written in a Terraform module using the Terraform language's type constraint syntax:
map(
object({
name = string
env = string
id = number
project = string
})
)

How to restructure data

I have a data of this format.
type PartsInfo struct {
Parts map[string]struct {
City string `yaml:"city"`
Services map[string]struct {
Disabled bool `yaml:"disabled"`
} `yaml:"services"`
} `yaml:"parts"`
}
I want to convert it into this format:
map[service]map[city][]parts where only not disabled status services need to be added. I have been trying different combinations but cant get it just the way i want.
I guess one thing I am not sure of is the destination format. Should I be using the map[service]map[city][]parts format, or would a struct be better ? I do not see how, but I have been told before the best way to pass data in go is using structs, not maps. Is that correct?
Is this what you want?
Go Playground: https://play.golang.org/p/N8mkD5pt1pD
package main
import "fmt"
type PartitionData struct {
Partitions map[string]Partition `yaml:"parts"`
}
type Partition struct {
City string `yaml:"city"`
Services map[string]map[string]struct {
Disabled bool `yaml:"disabled"`
} `yaml:"services"`
}
var testData = PartitionData{
Partitions: map[string]Partition{
"partition1": {City: "c1", Services: map[string]map[string]struct{
Disabled bool `yaml:"disabled"`
}{
"service1":{
"1":{
Disabled: true,
},
"2":{
Disabled: true,
},
},
"service2":{
"1":{
Disabled: true,
},
"2":{
Disabled: true,
},
},
}},
"partition2": {City: "c1", Services: map[string]map[string]struct{
Disabled bool `yaml:"disabled"`
}{
"service1":{
"1":{
Disabled: true,
},
"2":{
Disabled: true,
},
},
"service2":{
"1":{
Disabled: true,
},
"2":{
Disabled: true,
},
},
}},
},
}
func main() {
res:= make(map[string]map[string][]Partition)
for _,part := range testData.Partitions{
for serviceName :=range part.Services{
if _,found := res[serviceName];!found {
res[serviceName] = make(map[string][]Partition)
}
if _,found := res[serviceName][part.City];!found {
res[serviceName][part.City] = make([]Partition,0)
}
res[serviceName][part.City] = append(res[serviceName][part.City], part)
}
}
fmt.Println(res)
}

Accessing typeset in Golang from the terraform schema and iterate over the Map

I am trying to access a key element in Golang with the following schema via terraform config file:
"vehicles": {
Type: schema.TypeSet,
Optional: true,
MaxItems: 5,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"car": {
Type: schema.TypeList,
Optional: true,
MaxItems: 2,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"make": {
Type: schema.TypeString,
Optional: true,
},
"model": {
Type: schema.TypeString,
Optional: true,
},
},
},
},
},
},
}
In config file,
resource "type_test" "type_name" {
vehicles {
car {
make = "Toyota"
model = "Camry"
}
car {
make = "Nissan"
model = "Rogue"
}
}
}
I want to iterate over the list and access the vehicles map via Golang.
The terraform crashes with the below code:
vehicles_map, ok = d.getOK("vehicles")
if ok {
vehicleSet := vehicles_d.(*schema.Set)List()
for i, vehicle := range vehicleSet {
mdi, ok = vehicle.(map[string]interface{})
if ok {
log.Printf("%v", mdi["vehicles"].(map[string]interface{})["car"])
}
}
Crash Log:
2019-12-25T21 [DEBUG] plugin.terraform-provider: panic: interface conversion: interface {} is nil, not map[string]interface {}
for line "log.Printf("%v", mdi["vehicles"].(map[string]interface{})["car"])"
I want to print and access the each vehicles element in the config file, any help would be appreciated.
d.getOK("vehicles") already performs the indexing with "vehicles" key, which results in a *schema.Set. Calling its Set.List() method, you get a slice (of type []interface{}). Iterating over its elements will give you values that represent a car, modeled with type map[string]interface{}. So inside the loop you just have to type assert to this type, and not index again with "vehicles" nor with "car".
Something like this:
for i, vehicle := range vehicleSet {
car, ok := vehicle.(map[string]interface{})
if ok {
log.Printf("model: %v, make: %v\n", car["model"], car["make"])
}
}

Expressing a complex structure in Terraform and reading it in Go

Note: edited after a comment from #JimB
I am trying to build a new Terraform provider in Go. The resource that I need is a bit complex. It includes structures, arrays within structures, arrays and structures within arrays. When I run Terraform, it gives me errors, for example:
panic: Error reading level config: '' expected type 'string', got unconvertible type 'map[string]interface {}'. I can't figure out what I am doing wrong.
When I make the structures simple enough, they do work, but I need this resource and I'm sure there's a way to do it, and I'm just missing something perhaps trivial.
-- Here's the Terraform structure:
resource "struct" "my-struct-1" {
name = "MyFile"
complexstruct = [{
onebool = true
onearray = [{
name = "name-1"
value = "value-1"
}, {
name = "name-2"
value = "value-2"
}]
internalstruct = [{
attr1 = false
attr2 = "attribute"
}]
}]
array = [
{
attrib1 = "attrib1.1"
attrib2 = false
attrib3 = "attrib1.3"
},
{
attrib1 = "attrib2.1"
attrib2 = true
attrib3 = "attrib2.3"
}
]
}
-- Here is the Schema definition in go, as simplified as I could make it:
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Required: true,
},
"complexstruct": {
Type: schema.TypeList,
MaxItems: 1,
Optional: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"onebool": {
Type: schema.TypeBool,
Optional: true,
},
"onearray": {
Type: schema.TypeList,
Optional: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"name": {
Type: schema.TypeString,
Optional: true,
},
"value": {
Type: schema.TypeString,
Optional: true,
},
},
},
},
"internalstruct": {
Type: schema.TypeList,
MaxItems: 1,
Optional: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"attr1": {
Type: schema.TypeBool,
Optional: true,
},
"attr2": {
Type: schema.TypeString,
Optional: true,
},
},
},
},
},
},
},
"array": {
Type: schema.TypeList,
Optional: true,
Elem: map[string]*schema.Schema{
"attrib1": {
Type: schema.TypeString,
Optional: true,
},
"attrib2": {
Type: schema.TypeBool,
Optional: true,
},
"attrib3": {
Type: schema.TypeString,
Optional: true,
},
},
},
},
----- And lastly, here's the code that I am trying to use (however, I think the problem is before it starts with the code itself):
fname := d.Get("name").(string)
d.SetId(fname)
if _, ok := d.GetOk("complexstruct"); ok {
fc := d.Get("complexstruct").([]map[string]interface{})
myBool := fc[0]["onebool"].(bool)
myArray := fc[0]["onearray"].([]map[string]interface{})
type fcS struct {
Name string `json:"name"`
Value string `json:"value"`
}
fcs := make([]fcS, len(myArray))
for ifc := range myArray {
fcs[ifc].Name = myArray[ifc]["name"].(string)
fcs[ifc].Value = myArray[ifc]["value"].(string)
}
myInternalStruct := fc[0]["internalstruct"].([]map[string]interface{})
type misS struct {
Attr1 bool `json:"attr1"`
Attr2 string `json:"attr2"'`
}
var mis misS
mis.Attr1 = myInternalStruct[0]["attr1"].(bool)
mis.Attr2 = myInternalStruct[0]["attr2"].(string)
type myWholeStruct struct {
MyBool bool `json:"onebool"`
MyArray []fcS `json:"onearray"`
MyInter misS `json:"internalstruct"`
}
outp := myWholeStruct{
myBool,
fcs,
mis,
}
o, _ := json.Marshal(outp)
writeStringToFile(string(o), fname, false)
}
Well, I expect the create function to create a file with the name taken from the name attribute, and the data a JSON representation of the values of the other Terraform attributes. Instead I am getting errors as specified above.

Resources